00010000:00010000:2.0F:1782107176.965703:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0F:1782107177.417701:0:7431:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906600/0x7e3d6a8eb3a12d17 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7e3d6a8eb3a12d1e expref: -99 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1782107177.417730:0:7431:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906600/0x7e3d6a8eb3a12d17 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7e3d6a8eb3a12d1e expref: -99 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107177.417771:0:7431:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906600/0x7e3d6a8eb3a12d17 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12d1e expref: -99 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107177.417778:0:7431:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1782107177.417810:0:7431:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906600/0x7e3d6a8eb3a12d17 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12d1e expref: -99 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1782107177.417827:0:7431:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967310 is committed 00000020:00080000:0.0:1782107177.417840:0:7431:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa08b414b29c0 refcount 2 committed 0 result 0 batchid 3 00000020:00080000:0.0:1782107177.417844:0:7431:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffffa08b41f7c400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa08b414b23c0 00000020:00080000:0.0:1782107177.417847:0:7431:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].3 00000020:00080000:0.0:1782107177.417849:0:7431:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffffa08b41f7cf00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa08b41eff800 00000020:00080000:0.0:1782107177.417852:0:7431:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].3 00000020:00080000:0.0:1782107177.417860:0:7431:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa08b414886c0 refcount 2 committed 0 result 0 batchid 2 00000020:00080000:0.0:1782107177.417863:0:7431:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffffa08b41f7c300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa08b41488a80 00000020:00080000:0.0:1782107177.417866:0:7431:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].2 00000020:00080000:0.0:1782107177.417868:0:7431:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffffa08b41f7c800 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa08c7ba64a00 00000020:00080000:0.0:1782107177.417870:0:7431:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].2 00000020:00080000:0.0:1782107177.417878:0:7431:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0001: 1 batchid updated 00000100:00100000:3.0F:1782107177.417908:0:3659:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1782107177.417933:0:3659:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3659:1868674269155456:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0F:1782107177.417974:0:3644:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907800/0x7e3d6a8eb3a12d8e lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.417987:0:3644:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907800/0x7e3d6a8eb3a12d8e lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1782107177.417993:0:3659:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1782107177.417993:0:3644:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907800/0x7e3d6a8eb3a12d8e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.417999:0:3644:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42907800 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:1.0:1782107177.418003:0:3644:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907800/0x7e3d6a8eb3a12d8e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418010:0:3644:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907800/0x7e3d6a8eb3a12d8e lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418021:0:3644:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffffa08c7a609300) 00010000:00010000:1.0:1782107177.418024:0:3644:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1782107177.418028:0:3644:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907800/0x7e3d6a8eb3a12d8e lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418033:0:3644:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907800/0x7e3d6a8eb3a12d8e lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418043:0:3644:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906e00/0x7e3d6a8eb3a12d95 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418048:0:3644:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906e00/0x7e3d6a8eb3a12d95 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418053:0:3644:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906e00/0x7e3d6a8eb3a12d95 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418058:0:3644:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42906e00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:1.0:1782107177.418066:0:3644:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906e00/0x7e3d6a8eb3a12d95 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418077:0:3644:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906e00/0x7e3d6a8eb3a12d95 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418085:0:3644:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffffa08c7a609600) 00010000:00010000:1.0:1782107177.418088:0:3644:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1782107177.418090:0:3644:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906e00/0x7e3d6a8eb3a12d95 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107177.418092:0:9469:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269155456 00010000:00010000:1.0:1782107177.418096:0:3644:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906e00/0x7e3d6a8eb3a12d95 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107177.418098:0:9469:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269155456 00010000:00010000:1.0:1782107177.418124:0:3644:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12e21 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418131:0:3644:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12e21 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418137:0:3644:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12e21 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418141:0:3644:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42907e00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:1.0:1782107177.418144:0:3644:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12e21 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107177.418148:0:9469:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9 00010000:00010000:1.0:1782107177.418150:0:3644:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12e21 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418157:0:3644:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffffa08c7a609d00) 00000100:00100000:0.0:1782107177.418157:0:9469:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba51180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3659:x1868674269155456:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1782107177.418159:0:3644:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1782107177.418165:0:3644:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12e21 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107177.418170:0:3644:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12e21 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107177.418174:0:9469:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1782107177.418205:0:9469:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12d1e lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12d17 expref: 7 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107177.418224:0:9469:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5d0027 (ffffa08c43d2b000) 00010000:00010000:0.0:1782107177.418231:0:9469:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1782107177.418273:0:9469:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba51180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3659:x1868674269155456:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 118us (280us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107177.418281:0:9469:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9 00000100:00100000:1.0:1782107177.418353:0:3659:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3659:1868674269155456:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:2.0:1782107177.418418:0:7453:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3 committed batchid 1 00000004:00080000:2.0:1782107177.418528:0:7453:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa08b42fd9d80 with version 7 00000020:00080000:2.0:1782107177.418539:0:7453:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 1 cancel update log [0x200000401:0x3:0x0].1: rc = 0 00000020:00080000:2.0:1782107177.418578:0:7453:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 1 cancel update log [0x240000400:0x3:0x0].1: rc = 0 00000004:00080000:0.0:1782107177.418583:0:7445:0:(osp_trans.c:1330:osp_get_next_request()) ou ffffa08c453a7840 version 7 rpc_version 7 00000020:00080000:2.0:1782107177.418586:0:7453:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 1 00000020:00100000:0.0:1782107177.418627:0:7445:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1782107177.418635:0:7445:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa08b4155ce00 x1868674269155712/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782107177.418652:0:7445:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155ce00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269155712:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1782107177.418685:0:7445:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107177.418712:0:7445:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b41786d80 going to sleep for 11 seconds 00000100:00100000:3.0:1782107177.418909:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269155712 00000100:00100000:3.0:1782107177.418913:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269155712 00000100:00100000:3.0:1782107177.418947:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:3.0:1782107177.418954:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c51708450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:7445:x1868674269155712:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1782107177.419075:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c51708450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7445:x1868674269155712:12345-0@lo:1000:osp_up0-1.0 Request processed in 122us (388us total) trans 4294967361 rc 0/0 00000100:00100000:3.0:1782107177.419082:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:0.0:1782107177.419313:0:7445:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155ce00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269155712:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1782107177.989219:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782107178.391273:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1782107178.391363:0:8982:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246975488 00000100:00100000:1.0:1782107178.391373:0:8982:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246975488 00000100:00100000:1.0:1782107178.391535:0:8982:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:1.0:1782107178.391545:0:8982:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:34212:x1868674246975488:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1782107178.391566:0:8982:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1782107178.391606:0:8982:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12e44 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e31 expref: -99 pid: 8982 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.391674:0:8982:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea400/0x7e3d6a8eb3a12e4b lrc: 3/1,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8982 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1782107178.391738:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:1.0:1782107178.391755:0:8982:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea400/0x7e3d6a8eb3a12e4b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.391768:0:8982:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea400/0x7e3d6a8eb3a12e4b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1782107178.391859:0:8982:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 4096 00000004:00200000:1.0:1782107178.391864:0:8982:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1782107178.391882:0:8982:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea400/0x7e3d6a8eb3a12e4b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.391905:0:8982:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12e44 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e31 expref: 8 pid: 8982 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.391943:0:8982:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea400/0x7e3d6a8eb3a12e4b lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e31 expref: 7 pid: 8982 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.391952:0:8982:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffffa08c7a609300) 00010000:00010000:1.0:1782107178.391955:0:8982:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1782107178.391959:0:8982:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c412ea400, rc 0) 00000100:00100000:1.0:1782107178.392016:0:8982:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2eadce33-9ab8-4caf-b349-1b7b3db092bc+7:34212:x1868674246975488:12345-192.168.203.43@tcp:101:rm.0 Request processed in 472us (753us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107178.392023:0:8982:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:2.0:1782107178.400695:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1782107178.400791:0:18105:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246975616 00000100:00100000:1.0:1782107178.400800:0:18105:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246975616 00000100:00100000:1.0:1782107178.400883:0:18105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00100000:1.0:1782107178.400893:0:18105:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+7:34212:x1868674246975616:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1782107178.400911:0:18105:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1782107178.400991:0:18105:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea000/0x7e3d6a8eb3a12e52 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e38 expref: -99 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.401047:0:18105:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb800/0x7e3d6a8eb3a12e59 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.401061:0:18105:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb800/0x7e3d6a8eb3a12e59 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.401073:0:18105:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb800/0x7e3d6a8eb3a12e59 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1782107178.401111:0:18105:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1782107178.401122:0:18105:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1782107178.401219:0:18105:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea000/0x7e3d6a8eb3a12e52 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e38 expref: 9 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.401235:0:18105:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb800/0x7e3d6a8eb3a12e59 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e38 expref: 8 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.401244:0:18105:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffffa08c469afc00) 00010000:00010000:1.0:1782107178.401247:0:18105:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1782107178.401251:0:18105:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c412eb800, rc 0) 00000100:00100000:1.0:1782107178.402341:0:18105:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc21f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+8:34212:x1868674246975616:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1446us (1650us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107178.402351:0:18105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00100000:3.0:1782107178.410454:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1782107178.410634:0:6516:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246975872 00000100:00100000:1.0:1782107178.410644:0:6516:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246975872 00000100:00100000:1.0:1782107178.410702:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 179 00000100:00100000:1.0:1782107178.410712:0:6516:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+8:34212:x1868674246975872:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1782107178.410732:0:6516:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1782107178.410753:0:6516:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12e60 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e3f expref: -99 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1782107178.410877:0:6516:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 4096 00010000:00010000:1.0:1782107178.410973:0:6516:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12e60 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e3f expref: 10 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.410988:0:6516:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c469af700) 00010000:00010000:1.0:1782107178.410992:0:6516:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1782107178.410997:0:6516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12e60 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e3f expref: 10 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.411024:0:6516:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c412eb400, rc 0) 00000100:00100000:1.0:1782107178.411083:0:6516:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+9:34212:x1868674246975872:12345-192.168.203.43@tcp:101:rm.0 Request processed in 372us (635us total) trans 4294967311 rc 301/301 00000100:00100000:1.0:1782107178.411091:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 179 00000100:00100000:2.0:1782107178.417690:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107178.418026:0:6519:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246976000 00000100:00100000:3.0:1782107178.418035:0:6519:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246976000 00000100:00100000:3.0:1782107178.418088:0:6519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 27 00000100:00100000:3.0:1782107178.418100:0:6519:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c51158450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+9:34212:x1868674246976000:12345-192.168.203.43@tcp:37:rm.0 00000100:00100000:3.0:1782107178.425297:0:6519:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c51158450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+9:34212:x1868674246976000:12345-192.168.203.43@tcp:37:rm.0 Request processed in 7188us (7607us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107178.425321:0:6519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 27 00000100:00100000:3.0:1782107178.428735:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107178.429137:0:6520:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246976128 00000100:00100000:2.0:1782107178.429146:0:6520:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246976128 00000100:00100000:2.0:1782107178.429217:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 28 00000100:00100000:2.0:1782107178.429227:0:6520:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:2eadce33-9ab8-4caf-b349-1b7b3db092bc+8:34212:x1868674246976128:12345-192.168.203.43@tcp:37:rm.0 00000100:00080000:1.0:1782107178.430480:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1782107178.438131:0:6520:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:2eadce33-9ab8-4caf-b349-1b7b3db092bc+8:34212:x1868674246976128:12345-192.168.203.43@tcp:37:rm.0 Request processed in 8896us (9394us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107178.438144:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 28 00000100:00100000:1.0:1782107178.440634:0:3659:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7fc20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3659:1868674269155840:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1782107178.440694:0:3659:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107178.441545:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269155840 00000100:00100000:0.0:1782107178.441553:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269155840 00000100:00100000:0.0:1782107178.441643:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:0.0:1782107178.441653:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7bbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3659:x1868674269155840:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1782107178.441689:0:7835:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446219776 avail 1299980288 00000100:00100000:2.0:1782107178.443323:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107178.443358:0:6519:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246976384 00000100:00100000:2.0:1782107178.443365:0:6519:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246976384 00000100:00100000:2.0:1782107178.443428:0:6519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 29 00000100:00100000:2.0:1782107178.443437:0:6519:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc21880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+9:34212:x1868674246976384:12345-192.168.203.43@tcp:35:rm.0 00000100:00100000:2.0:1782107178.443635:0:6519:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc21880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+10:34212:x1868674246976384:12345-192.168.203.43@tcp:35:rm.0 Request processed in 198us (317us total) trans 4294967312 rc 0/0 00000100:00100000:2.0:1782107178.443642:0:6519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 29 00000100:00100000:0.0:1782107178.445174:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7bbf8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3659:x1868674269155840:12345-0@lo:41:osp-pre-0-1.0 Request processed in 3516us (4472us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107178.445185:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:1.0:1782107178.447969:0:3659:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7fc20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3659:1868674269155840:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1782107178.451496:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1782107178.451674:0:8982:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246976640 00000100:00100000:1.0:1782107178.451694:0:8982:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246976640 00000100:00100000:1.0:1782107178.451751:0:8982:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 180 00000100:00100000:1.0:1782107178.451761:0:8982:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2eadce33-9ab8-4caf-b349-1b7b3db092bc+10:34212:x1868674246976640:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1782107178.451788:0:8982:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1782107178.451810:0:8982:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12e6e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e46 expref: -99 pid: 8982 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.451950:0:8982:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea000/0x7e3d6a8eb3a12e75 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8982 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.451964:0:8982:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea000/0x7e3d6a8eb3a12e75 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x3:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.451974:0:8982:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea000/0x7e3d6a8eb3a12e75 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x3:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8982 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1782107178.451996:0:8982:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 4096 00010000:00010000:1.0:1782107178.452082:0:8982:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12e6e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e46 expref: 13 pid: 8982 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.452100:0:8982:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea000/0x7e3d6a8eb3a12e75 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x3:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e46 expref: 12 pid: 8982 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.452108:0:8982:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffffa08c7a609300) 00010000:00010000:1.0:1782107178.452111:0:8982:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1782107178.452114:0:8982:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c412ea000, rc 0) 00000100:00100000:1.0:1782107178.452283:0:8982:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2eadce33-9ab8-4caf-b349-1b7b3db092bc+12:34212:x1868674246976640:12345-192.168.203.43@tcp:101:rm.0 Request processed in 523us (795us total) trans 4294967313 rc 0/0 00000100:00100000:1.0:1782107178.452290:0:8982:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 180 00000100:00100000:2.0:1782107178.464810:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107178.465060:0:6518:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246976768 00000100:00100000:3.0:1782107178.465071:0:6518:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246976768 00000100:00100000:3.0:1782107178.465138:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:3.0:1782107178.465151:0:6518:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc22680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+8:34212:x1868674246976768:12345-192.168.203.43@tcp:41:rm.0 00000004:00000024:3.0:1782107178.465219:0:6518:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 7847059456 avail 7384014848 00000100:00100000:3.0:1782107178.465299:0:6518:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc22680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+8:34212:x1868674246976768:12345-192.168.203.43@tcp:41:rm.0 Request processed in 152us (496us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107178.465316:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:3.0:1782107178.470630:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1782107178.470814:0:7836:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246976896 00000100:00100000:0.0:1782107178.470833:0:7836:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246976896 00000100:00100000:0.0:1782107178.470989:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:0.0:1782107178.470999:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+12:34212:x1868674246976896:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1782107178.471021:0:7836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1782107178.471040:0:7836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943400/0x7e3d6a8eb3a12e83 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e4d expref: -99 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471096:0:7836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 3/1,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471108:0:7836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471118:0:7836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471131:0:7836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471138:0:7836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471143:0:7836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471202:0:7836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942800/0x7e3d6a8eb3a12e98 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471221:0:7836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942800/0x7e3d6a8eb3a12e98 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471228:0:7836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942800/0x7e3d6a8eb3a12e98 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1782107178.471269:0:7836:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x4:0x0]: returning size 0 00000004:00200000:0.0:1782107178.471272:0:7836:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1782107178.471290:0:7836:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942800/0x7e3d6a8eb3a12e98 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471300:0:7836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942800/0x7e3d6a8eb3a12e98 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471318:0:7836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471322:0:7836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 2/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471327:0:7836:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471330:0:7836:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c7b943000 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:0.0:1782107178.471333:0:7836:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471337:0:7836:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471345:0:7836:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffffa08c469afc00) 00010000:00010000:0.0:1782107178.471347:0:7836:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1782107178.471350:0:7836:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471353:0:7836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943000/0x7e3d6a8eb3a12e8a lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471392:0:7836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471448:0:7836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471454:0:7836:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471460:0:7836:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c7b942000 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:0.0:1782107178.471463:0:7836:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471469:0:7836:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471477:0:7836:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffffa08b42efa200) 00010000:00010000:0.0:1782107178.471479:0:7836:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1782107178.471482:0:7836:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471488:0:7836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942000/0x7e3d6a8eb3a12e91 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7836 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471522:0:7836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b943400/0x7e3d6a8eb3a12e83 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e4d expref: 14 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471534:0:7836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942800/0x7e3d6a8eb3a12e98 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e4d expref: 13 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.471551:0:7836:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffffa08b42efa600) 00010000:00010000:0.0:1782107178.471553:0:7836:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1782107178.471555:0:7836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c7b942800, rc 0) 00000100:00100000:0.0:1782107178.471610:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+13:34212:x1868674246976896:12345-192.168.203.43@tcp:101:rm.0 Request processed in 612us (990us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107178.471617:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:0.0:1782107178.474319:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107178.474449:0:6517:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246977152 00000100:00100000:3.0:1782107178.474459:0:6517:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246977152 00000100:00100000:3.0:1782107178.474502:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 183 00000100:00100000:3.0:1782107178.474510:0:6517:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2eadce33-9ab8-4caf-b349-1b7b3db092bc+13:34212:x1868674246977152:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:3.0:1782107178.474554:0:6517:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1782107178.474577:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c7b942800/0x7e3d6a8eb3a12e98 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e4d expref: 13 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.474594:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffffa08b42efa600) 00010000:00010000:3.0:1782107178.474606:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb800/0x7e3d6a8eb3a12e59 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e38 expref: 12 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.474615:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffffa08c469afc00) 00010000:00010000:3.0:1782107178.474622:0:6517:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1782107178.474642:0:6517:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 3/0,1 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.474654:0:6517:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.474664:0:6517:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.474677:0:6517:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.476172:0:6517:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.476725:0:6517:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.476817:0:6517:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.476825:0:6517:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.476831:0:6517:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477039:0:6517:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477049:0:6517:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477057:0:6517:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477064:0:6517:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477070:0:6517:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477076:0:6517:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477095:0:6517:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1782107178.477115:0:6517:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477125:0:6517:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477131:0:6517:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477136:0:6517:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42906800 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:3.0:1782107178.477140:0:6517:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477147:0:6517:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477156:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffffa08c7a609200) 00010000:00010000:3.0:1782107178.477159:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1782107178.477163:0:6517:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 2/0,0 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477168:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ead lrc: 0/0,0 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477183:0:6517:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477188:0:6517:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477193:0:6517:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477198:0:6517:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477219:0:6517:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42906200 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:3.0:1782107178.477222:0:6517:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477228:0:6517:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477235:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffffa08c7a609800) 00010000:00010000:3.0:1782107178.477237:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107178.477240:0:6517:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 3/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477246:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906200/0x7e3d6a8eb3a12e9f lrc: 0/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1782107178.477258:0:6517:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000008691f534 reply state 0000000017ae608c transno 4294967314 ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477268:0:6517:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.477274:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffffa08c7a609400) 00010000:00010000:3.0:1782107178.477276:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1782107178.477278:0:6517:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1782107178.477365:0:6517:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2eadce33-9ab8-4caf-b349-1b7b3db092bc+13:34212:x1868674246977152:12345-192.168.203.43@tcp:36:rm.0 Request processed in 2852us (3064us total) trans 4294967314 rc 0/0 00000100:00100000:3.0:1782107178.477374:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 183 00010000:00010000:2.0:1782107178.477759:0:6504:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1782107178.477766:0:6504:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.477772:0:6504:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42906000 already unused, calling callback (ffffffffc0e666f0) 00010000:00010000:2.0:1782107178.477775:0:6504:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.477789:0:6504:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.477799:0:6504:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffffa08c7a609200) 00010000:00010000:2.0:1782107178.477801:0:6504:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1782107178.477805:0:6504:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.477810:0:6504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12eb4 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.477824:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782107178.477826:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782107178.477827:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782107178.487103:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107178.487143:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107178.487220:0:6519:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246977536 00000100:00100000:3.0:1782107178.487231:0:6519:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246977536 00000100:00100000:3.0:1782107178.487287:0:6519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 30 00000100:00100000:3.0:1782107178.487309:0:6519:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc22a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+13:34212:x1868674246977536:12345-192.168.203.43@tcp:35:rm.0 00000100:00100000:3.0:1782107178.487543:0:6519:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc22a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+14:34212:x1868674246977536:12345-192.168.203.43@tcp:35:rm.0 Request processed in 235us (398us total) trans 4294967315 rc 0/0 00000100:00100000:3.0:1782107178.487550:0:6519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 30 00000100:00100000:0.0:1782107178.488463:0:6503:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246977280 00000100:00100000:0.0:1782107178.488471:0:6503:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246977280 00000100:00100000:0.0:1782107178.488522:0:6503:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 10 00000100:00100000:0.0:1782107178.488531:0:6503:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc20a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+8:2414:x1868674246977280:12345-192.168.203.43@tcp:103:rm.0 00010000:00010000:0.0:1782107178.488550:0:6503:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1782107178.488573:0:6503:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c7b942c00/0x7e3d6a8eb3a12dcd lrc: 0/0,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e00 expref: 8 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.488591:0:6503:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffffa08c447b8e00) 00010000:00010000:0.0:1782107178.488601:0:6503:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1782107178.488638:0:6503:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc20a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+7:2414:x1868674246977280:12345-192.168.203.43@tcp:103:rm.0 Request processed in 109us (1544us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107178.488645:0:6503:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 10 00000100:00100000:2.0:1782107178.493299:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107178.493431:0:6518:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246977792 00000100:00100000:3.0:1782107178.493441:0:6518:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246977792 00000100:00100000:3.0:1782107178.493499:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 184 00000100:00100000:3.0:1782107178.493510:0:6518:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc22d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+14:34212:x1868674246977792:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:3.0:1782107178.493574:0:6518:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1782107178.493600:0:6518:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea000/0x7e3d6a8eb3a12e75 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e46 expref: 14 pid: 8982 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.493621:0:6518:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea400/0x7e3d6a8eb3a12e4b lrc: 0/0,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e31 expref: 13 pid: 8982 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.493633:0:6518:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12ddb lrc: 0/0,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e07 expref: 12 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.493642:0:6518:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffffa08c7a609300) 00010000:00010000:3.0:1782107178.493652:0:6518:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1782107178.493694:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.493701:0:6518:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.493711:0:6518:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1782107178.493746:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7ba51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269155968:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1782107178.493791:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782107178.493852:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08c81af0480 going to sleep for 11 seconds 00000100:00100000:0.0:1782107178.493918:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269155968 00000100:00100000:0.0:1782107178.493923:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269155968 00000100:00100000:0.0:1782107178.493959:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:0.0:1782107178.493965:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6518:x1868674269155968:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1782107178.493975:0:6522:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1782107178.493994:0:6522:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7e3d6a8eb3a12ebb expref: -99 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.494016:0:6522:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907600/0x7e3d6a8eb3a12db1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3df9 expref: 6 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.494024:0:6522:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ebb expref: 8 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.494045:0:6522:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907600/0x7e3d6a8eb3a12db1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3df9 expref: 6 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.494058:0:6522:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907600/0x7e3d6a8eb3a12db1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3df9 expref: 6 pid: 6516 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107178.494068:0:6522:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155d880 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6522:1868674269156096:192.168.203.43@tcp:104: 00000100:00100000:0.0:1782107178.494130:0:6522:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08c7a51a180 going to sleep for 11 seconds 00000100:00100000:3.0:1782107178.500373:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107178.500469:0:6522:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155d880 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6522:1868674269156096:192.168.203.43@tcp:104: 00010000:00010000:3.0:1782107178.500499:0:6522:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ebb expref: 8 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.500512:0:6522:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:3.0:1782107178.500515:0:6522:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107178.500519:0:6522:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ebb expref: 8 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.500529:0:6522:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c40030000, rc 0) 00000100:00100000:3.0:1782107178.500580:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6518:x1868674269155968:12345-0@lo:101:mdt00_002.0 Request processed in 6615us (6788us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107178.500589:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:3.0:1782107178.500637:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7ba51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269155968:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1782107178.500653:0:6518:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.500663:0:6518:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107178.500739:0:9470:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246977920 00000100:00100000:2.0:1782107178.500745:0:9470:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246977920 00010000:00010000:2.0:1782107178.500774:0:9470:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907600/0x7e3d6a8eb3a12db1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3df9 expref: 7 pid: 6516 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107178.500806:0:9470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 70 00000100:00100000:2.0:1782107178.500815:0:9470:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba52680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:2eadce33-9ab8-4caf-b349-1b7b3db092bc+7:2412:x1868674246977920:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1782107178.500829:0:9470:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1782107178.500834:0:9470:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907600/0x7e3d6a8eb3a12db1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3df9 expref: 7 pid: 6516 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.500863:0:9470:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907600/0x7e3d6a8eb3a12db1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3df9 expref: 7 pid: 6516 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.500884:0:9470:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907600/0x7e3d6a8eb3a12db1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3df9 expref: 7 pid: 6516 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.500897:0:9470:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:2.0:1782107178.500901:0:9470:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ebb expref: 7 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.500911:0:9470:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ebb expref: 7 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.500919:0:9470:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ebb expref: 7 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.500959:0:9470:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ebb expref: 7 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107178.500974:0:9470:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7fc21180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:9470:1868674269156224:0@lo:105: 00000100:00100000:2.0:1782107178.501010:0:9470:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107178.501040:0:9470:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9840 going to sleep for 11 seconds 00000100:00100000:2.0:1782107178.501053:0:6499:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269156224 00000100:00100000:2.0:1782107178.501057:0:6499:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269156224 00000100:00100000:2.0:1782107178.501085:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:2.0:1782107178.501091:0:6499:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c60850380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9470:x1868674269156224:12345-0@lo:105: 00010000:00010000:2.0:1782107178.501107:0:6499:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501115:0:6499:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501122:0:6499:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501130:0:6499:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501136:0:6499:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501145:0:6499:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501163:0:6499:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa08b42906200) 00000100:00100000:2.0:1782107178.501192:0:6499:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c60850380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9470:x1868674269156224:12345-0@lo:105: Request processed in 101us (181us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107178.501198:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00100000:2.0:1782107178.501240:0:9470:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7fc21180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:9470:1868674269156224:0@lo:105: 00010000:00010000:2.0:1782107178.501250:0:9470:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1782107178.501285:0:9470:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba52680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:2412:x1868674246977920:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 472us (921us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107178.501291:0:9470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 70 00010000:00010000:2.0:1782107178.501395:0:6518:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501438:0:6518:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1782107178.501458:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1782107178.501491:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501499:0:6518:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501508:0:6518:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107178.501523:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c60853800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156352:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1782107178.501548:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107178.501565:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9840 going to sleep for 11 seconds 00000100:00100000:2.0:1782107178.501578:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269156352 00000100:00100000:2.0:1782107178.501581:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269156352 00000100:00100000:2.0:1782107178.501605:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1782107178.501611:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c60852a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6518:x1868674269156352:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1782107178.501625:0:7835:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1782107178.501689:0:7835:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419a00/0x7e3d6a8eb3a12ed0 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7e3d6a8eb3a12ec9 expref: -99 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501708:0:7835:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419a00/0x7e3d6a8eb3a12ed0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ec9 expref: 9 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501731:0:7835:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419a00/0x7e3d6a8eb3a12ed0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ec9 expref: 9 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501738:0:7835:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffffa08c43d2b300) 00010000:00010000:2.0:1782107178.501741:0:7835:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1782107178.501743:0:7835:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b43419a00, rc 0) 00000100:00100000:2.0:1782107178.501772:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c60852a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269156352:12345-0@lo:101:mdt00_002.0 Request processed in 161us (224us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107178.501778:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1782107178.501806:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c60853800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156352:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1782107178.501816:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12ed0 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.501824:0:6518:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ed0 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1782107178.501835:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1782107178.501864:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c60853800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156480:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1782107178.501888:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107178.501901:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9240 going to sleep for 11 seconds 00000100:00100000:2.0:1782107178.501913:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269156480 00000100:00100000:2.0:1782107178.501916:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269156480 00000100:00100000:2.0:1782107178.501937:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:2.0:1782107178.501943:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc21180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269156480:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1782107178.501995:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc21180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269156480:12345-0@lo:1000:mdt00_002.0 Request processed in 52us (106us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107178.502007:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:2.0:1782107178.502039:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c60853800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156480:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1782107178.502078:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c60853800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156608:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1782107178.502102:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107178.502122:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9240 going to sleep for 11 seconds 00000100:00100000:0.0:1782107178.502215:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269156608 00000100:00100000:0.0:1782107178.502221:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269156608 00000100:00100000:0.0:1782107178.502252:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:0.0:1782107178.502260:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b434d7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269156608:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1782107178.502328:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b434d7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269156608:12345-0@lo:1000:mdt00_002.0 Request processed in 70us (226us total) trans 0 rc 1/1 00000100:00100000:0.0:1782107178.502335:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:2.0:1782107178.504557:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c60853800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156608:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1782107178.504643:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.504653:0:6518:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.504662:0:6518:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107178.504683:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b434d4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156736:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1782107178.504733:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107178.504761:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9840 going to sleep for 11 seconds 00000100:00100000:3.0:1782107178.504832:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269156736 00000100:00100000:3.0:1782107178.504839:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269156736 00000100:00100000:3.0:1782107178.504900:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:3.0:1782107178.504909:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b434d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269156736:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1782107178.504933:0:7835:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1782107178.504948:0:7835:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ede lrc: 2/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7e3d6a8eb3a12ed7 expref: -99 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.504973:0:7835:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ede lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ed7 expref: 10 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.504987:0:7835:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ede lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ed7 expref: 10 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.504995:0:7835:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffffa08c7a609800) 00010000:00010000:3.0:1782107178.504998:0:7835:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1782107178.505002:0:7835:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b42906800, rc 0) 00000100:00100000:3.0:1782107178.505047:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b434d5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6518:x1868674269156736:12345-0@lo:101:mdt00_002.0 Request processed in 140us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107178.505055:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:1.0:1782107178.505185:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b434d4700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156736:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1782107178.505197:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505217:0:6518:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505241:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505249:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505257:0:6518:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505286:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 3/0,1 mode: --/EX res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505293:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505301:0:6518:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505322:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505328:0:6518:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.505335:0:6518:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107178.505348:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156864:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1782107178.505375:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107178.505426:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b43408180 going to sleep for 11 seconds 00000100:00100000:3.0:1782107178.505701:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269156864 00000100:00100000:3.0:1782107178.505705:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269156864 00000100:00100000:3.0:1782107178.505738:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:3.0:1782107178.505744:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7bbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6518:x1868674269156864:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1782107178.505753:0:6521:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1782107178.505763:0:6521:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 2/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7e3d6a8eb3a12ef3 expref: -99 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.505781:0:6521:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eb000/0x7e3d6a8eb3a12df7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e15 expref: 5 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.505803:0:6521:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 4/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 11 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.505825:0:6521:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eb000/0x7e3d6a8eb3a12df7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e15 expref: 5 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.505839:0:6521:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eb000/0x7e3d6a8eb3a12df7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e15 expref: 5 pid: 6517 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1782107178.505850:0:6521:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7ba50000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6521:1868674269156992:192.168.203.43@tcp:104: 00000100:00100000:3.0:1782107178.505910:0:6521:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08c81af0480 going to sleep for 11 seconds 00000100:00100000:2.0:1782107178.510491:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107178.510731:0:6521:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7ba50000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6521:1868674269156992:192.168.203.43@tcp:104: 00010000:00010000:3.0:1782107178.510766:0:6521:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 3/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 11 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.510779:0:6521:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffffa08c43d2b400) 00010000:00010000:3.0:1782107178.510782:0:6521:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1782107178.510787:0:6521:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 3/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 11 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.510797:0:6521:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 3/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 11 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.510805:0:6521:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b42906000, rc 0) 00000100:00100000:3.0:1782107178.510867:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7bbf8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269156864:12345-0@lo:101:mdt00_002.0 Request processed in 5122us (5490us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107178.510888:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00100000:2.0:1782107178.510977:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269156864:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1782107178.510993:0:6518:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.511003:0:6518:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107178.511041:0:6501:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246978048 00000100:00100000:1.0:1782107178.511049:0:6501:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246978048 00010000:00010000:1.0:1782107178.511078:0:6501:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eb000/0x7e3d6a8eb3a12df7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e15 expref: 6 pid: 6517 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107178.511151:0:6501:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 71 00000100:00100000:1.0:1782107178.511161:0:6501:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b434d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:2413:x1868674246978048:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1782107178.511175:0:6501:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782107178.511187:0:6501:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eb000/0x7e3d6a8eb3a12df7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e15 expref: 6 pid: 6517 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.511199:0:6501:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eb000/0x7e3d6a8eb3a12df7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e15 expref: 6 pid: 6517 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.511231:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eb000/0x7e3d6a8eb3a12df7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e15 expref: 6 pid: 6517 timeout: 714 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.511244:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffffa08c43d2b400) 00010000:00010000:1.0:1782107178.511248:0:6501:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 2/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 10 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.511258:0:6501:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 10 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.511267:0:6501:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 10 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.511299:0:6501:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 10 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107178.511314:0:6501:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7bbf8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6501:1868674269157120:0@lo:105: 00000100:00100000:1.0:1782107178.511351:0:6501:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107178.511391:0:6501:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434080c0 going to sleep for 11 seconds 00000100:00100000:2.0:1782107178.511479:0:6499:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269157120 00000100:00100000:2.0:1782107178.511489:0:6499:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269157120 00000100:00100000:2.0:1782107178.511520:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 50 00000100:00100000:2.0:1782107178.511527:0:6499:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7bbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6501:x1868674269157120:12345-0@lo:105: 00010000:00010000:2.0:1782107178.511535:0:6499:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.511544:0:6499:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.511552:0:6499:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.511560:0:6499:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.511566:0:6499:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.511573:0:6499:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.511594:0:6499:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa08c412eba00) 00000100:00100000:2.0:1782107178.511628:0:6499:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7bbfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6501:x1868674269157120:12345-0@lo:105: Request processed in 101us (277us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107178.511634:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 50 00000100:00100000:1.0:1782107178.511739:0:6501:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7bbf8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6501:1868674269157120:0@lo:105: 00010000:00010000:1.0:1782107178.511751:0:6501:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782107178.511792:0:6501:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b434d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+5:2413:x1868674246978048:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 633us (1310us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107178.511805:0:6501:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 71 00010000:00010000:0.0:1782107178.512572:0:6518:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.512582:0:6518:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107178.513769:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269157248:0@lo:403:mdt00_002.0 00000100:00100000:0.0:1782107178.513832:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107178.513908:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08c7a51ac00 going to sleep for 11 seconds 00000100:00100000:3.0:1782107178.514181:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269157248 00000100:00100000:3.0:1782107178.514189:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269157248 00000100:00100000:3.0:1782107178.514279:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:3.0:1782107178.514290:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269157248:12345-0@lo:403:mdt00_002.0 00000100:00100000:3.0:1782107178.516778:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269157248:12345-0@lo:403:mdt00_002.0 Request processed in 2483us (2937us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107178.516791:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:0.0:1782107178.516863:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269157248:0@lo:403:mdt00_002.0 00000040:00080000:0.0:1782107178.520286:0:6518:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].4, 936 off38944 00000040:00080000:0.0:1782107178.520660:0:6518:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].4, 1016 off39504 00000004:00080000:0.0:1782107178.520668:0:6518:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa08c7a51ac00 with version 8 00000004:00080000:2.0:1782107178.520738:0:7445:0:(osp_trans.c:1330:osp_get_next_request()) ou ffffa08c453a7840 version 8 rpc_version 8 00000020:00100000:2.0:1782107178.520770:0:7445:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1782107178.520779:0:7445:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa08b434d4000 x1868674269157504/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2592/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782107178.520812:0:7445:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b434d4000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269157504:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1782107178.520873:0:7445:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107178.520905:0:7445:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9240 going to sleep for 11 seconds 00000100:00100000:3.0:1782107178.521051:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269157504 00000100:00100000:3.0:1782107178.521058:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269157504 00000100:00100000:3.0:1782107178.521105:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:3.0:1782107178.521113:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b434d4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7445:x1868674269157504:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1782107178.521731:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b434d4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7445:x1868674269157504:12345-0@lo:1000:osp_up0-1.0 Request processed in 616us (855us total) trans 4294967362 rc 0/0 00000100:00100000:3.0:1782107178.521738:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1782107178.521810:0:7445:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b434d4000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269157504:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1782107178.521861:0:6518:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 4096 00010000:00010000:0.0:1782107178.521881:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.521890:0:6518:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.521898:0:6518:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.521904:0:6518:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c412ea600 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:0.0:1782107178.521908:0:6518:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.521914:0:6518:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522042:0:6518:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffffa08c469af000) 00010000:00010000:0.0:1782107178.522046:0:6518:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1782107178.522051:0:6518:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 2/0,0 mode: --/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522068:0:6518:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ea600/0x7e3d6a8eb3a12eec lrc: 0/0,0 mode: --/EX res: [0x240000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522088:0:6518:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522095:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522101:0:6518:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522151:0:6518:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1782107178.522158:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000401:0x5:0x0] 00010000:00010000:0.0:1782107178.522164:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522170:0:6518:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522175:0:6518:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522179:0:6518:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c412eb400 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:0.0:1782107178.522182:0:6518:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522187:0:6518:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522213:0:6518:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffffa08c469af700) 00010000:00010000:0.0:1782107178.522215:0:6518:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1782107178.522219:0:6518:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522224:0:6518:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eb400/0x7e3d6a8eb3a12ee5 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522236:0:6518:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522244:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.522249:0:6518:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522277:0:6505:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1782107178.522283:0:6505:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522290:0:6505:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c412eba00 already unused, calling callback (ffffffffc1a15ee0) 00010000:00010000:3.0:1782107178.522294:0:6505:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1782107178.522300:0:6505:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522325:0:6505:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522337:0:6505:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1782107178.522370:0:6505:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522377:0:6505:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eba00/0x7e3d6a8eb3a12ef3 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12efa expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522390:0:6505:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1782107178.522393:0:6505:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522432:0:6505:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b43418e00 already unused, calling callback (ffffffffc1a15ee0) 00010000:00010000:3.0:1782107178.522435:0:6505:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1782107178.522442:0:6505:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522458:0:6505:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 4/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522464:0:6505:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1782107178.522470:0:6505:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 3/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107178.522477:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782107178.522478:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782107178.522480:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782107178.522546:0:3660:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7ba50000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269157632:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782107178.522586:0:3660:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107178.522616:0:3660:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782107178.522622:0:3660:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7ba50e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269157760:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782107178.522640:0:3660:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107178.522667:0:9470:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269157632 00000100:00100000:1.0:1782107178.522671:0:9470:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269157632 00000100:00100000:1.0:1782107178.522703:0:9470:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269157760 00000100:00100000:1.0:1782107178.522706:0:9470:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269157760 00000100:00100000:1.0:1782107178.522725:0:9470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11 00000100:00100000:1.0:1782107178.522732:0:9470:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c4abba850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+13:3660:x1868674269157632:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1782107178.522741:0:9470:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782107178.522756:0:9470:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906000/0x7e3d6a8eb3a12efa lrc: 0/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ef3 expref: 13 pid: 6521 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.522767:0:9470:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffffa08c43d2b400) 00010000:00010000:1.0:1782107178.522773:0:9470:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782107178.522800:0:9470:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c4abba850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3660:x1868674269157632:12345-0@lo:103:ldlm_bl.0 Request processed in 68us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107178.522805:0:9470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11 00000100:00100000:1.0:1782107178.522841:0:9470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12 00000100:00100000:1.0:1782107178.522846:0:9470:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7bbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3660:x1868674269157760:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1782107178.522852:0:9470:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782107178.522861:0:9470:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906800/0x7e3d6a8eb3a12ede lrc: 0/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ed7 expref: 11 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107178.522882:0:9470:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffffa08c7a609800) 00010000:00010000:1.0:1782107178.522888:0:9470:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782107178.522911:0:9470:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7bbf8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3660:x1868674269157760:12345-0@lo:103:ldlm_bl.0 Request processed in 65us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107178.522917:0:9470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12 00000100:00100000:1.0:1782107178.523184:0:3660:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7ba50e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269157760:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782107178.523192:0:3660:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7ba50000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269157632:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1782107178.523370:0:6518:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1782107178.523377:0:6518:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43418e00/0x7e3d6a8eb3a12ed7 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12ede expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1782107178.523420:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1782107178.523425:0:6518:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.523432:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.523438:0:6518:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.527111:0:6504:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1782107178.527125:0:6504:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.527139:0:6504:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42906200 already unused, calling callback (ffffffffc1a15ee0) 00010000:00010000:2.0:1782107178.527146:0:6504:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1782107178.527154:0:6504:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.527215:0:6504:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.527222:0:6504:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1782107178.527255:0:6504:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.527267:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782107178.527269:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782107178.527271:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782107178.527345:0:3658:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b434d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3658:1868674269157888:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782107178.527431:0:3658:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107178.527517:0:6501:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269157888 00000100:00100000:2.0:1782107178.527525:0:6501:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269157888 00000100:00100000:2.0:1782107178.527587:0:6501:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13 00000100:00100000:2.0:1782107178.527620:0:6501:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7bbf9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3658:x1868674269157888:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1782107178.527633:0:6501:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1782107178.527653:0:6501:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12ec2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ebb expref: 10 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.527669:0:6501:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:2.0:1782107178.527674:0:6501:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1782107178.527709:0:6501:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7bbf9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3658:x1868674269157888:12345-0@lo:103:ldlm_bl.0 Request processed in 91us (278us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107178.527717:0:6501:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13 00000100:00100000:1.0:1782107178.527788:0:3658:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b434d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3658:1868674269157888:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1782107178.527898:0:6518:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1782107178.527905:0:6518:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b42906200/0x7e3d6a8eb3a12ebb lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12ec2 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107178.528042:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1782107178.528126:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1782107178.528136:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12ed0 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107178.528146:0:6518:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12ed0 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107178.528229:0:6518:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc22d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+12:34212:x1868674246977792:12345-192.168.203.43@tcp:36:rm.0 Request processed in 34720us (34939us total) trans 4294967316 rc 0/0 00000100:00100000:0.0:1782107178.528238:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 184 00000100:00100000:2.0:1782107178.949015:0:3658:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1782107178.949037:0:3658:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3658:1868674269158016:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1782107178.949099:0:3658:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107178.949211:0:8402:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269158016 00000100:00100000:0.0:1782107178.949219:0:8402:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269158016 00000100:00100000:0.0:1782107178.949279:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1782107178.949290:0:8402:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b434d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3658:x1868674269158016:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1782107178.949317:0:8402:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1782107178.949378:0:8402:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b434d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3658:x1868674269158016:12345-0@lo:13:osp-pre-0-1.0 Request processed in 90us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107178.949385:0:8402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00080000:2.0:1782107178.949470:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1782107178.949507:0:3658:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3658:1868674269158016:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1782107178.951486:0:3661:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1782107178.951505:0:3661:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7ba53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3661:1868674269158144:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1782107178.951566:0:3661:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782107178.951636:0:17563:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269158144 00000100:00100000:3.0:1782107178.951645:0:17563:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269158144 00000100:00080000:2.0:1782107178.951770:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1782107178.951848:0:17563:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:3.0:1782107178.951857:0:17563:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c60853800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3661:x1868674269158144:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1782107178.951878:0:17563:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1782107178.951980:0:3661:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7ba53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3661:1868674269158144:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782107178.951990:0:17563:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c60853800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3661:x1868674269158144:12345-0@lo:13:osp-pre-0-0.0 Request processed in 133us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107178.951999:0:17563:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00010000:00010000:2.0:1782107179.011583:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782107179.287738:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107179.287781:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107179.287816:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1782107179.287879:0:6516:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246978304 00000100:00100000:0.0:1782107179.287879:0:7836:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246978432 00000100:00100000:0.0:1782107179.287889:0:7836:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246978432 00000100:00100000:1.0:1782107179.287892:0:6516:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246978304 00000100:00100000:0.0:1782107179.287949:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 621 00000100:00100000:0.0:1782107179.287960:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41564e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+12:2414:x1868674246978432:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1782107179.288065:0:6509:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246978176 00000100:00100000:3.0:1782107179.288069:0:6509:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246978176 00000100:00100000:0.0:1782107179.288071:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41564e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+12:2414:x1868674246978432:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 114us (290us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107179.288078:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 621 00000100:00100000:3.0:1782107179.288097:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 427 00000100:00100000:0.0:1782107179.288097:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 622 00000100:00100000:0.0:1782107179.288103:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41566a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+5:2414:x1868674246978304:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1782107179.288104:0:6509:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c42d5dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:87d3e82f-e99c-4ac6-accd-6b70667f4a2e+9:2414:x1868674246978176:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1782107179.288127:0:6509:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c42d5dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:87d3e82f-e99c-4ac6-accd-6b70667f4a2e+9:2414:x1868674246978176:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107179.288127:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41566a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+5:2414:x1868674246978304:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107179.288133:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 622 00000100:00100000:3.0:1782107179.288134:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 427 00000100:00100000:1.0:1782107179.459755:0:3661:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3661:1868674269158272:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782107179.459826:0:3661:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107179.459856:0:3658:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b41565f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3658:1868674269158400:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1782107179.459908:0:3658:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107179.459940:0:17563:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269158272 00000100:00100000:1.0:1782107179.459948:0:17563:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269158272 00000100:00100000:1.0:1782107179.459999:0:17563:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269158400 00000100:00100000:1.0:1782107179.460002:0:17563:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269158400 00000100:00100000:1.0:1782107179.460022:0:17563:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1782107179.460038:0:17563:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7bbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3661:x1868674269158272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782107179.460064:0:17563:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1782107179.460107:0:17563:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7bbfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3661:x1868674269158272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107179.460114:0:17563:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1782107179.460138:0:17563:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:1.0:1782107179.460145:0:17563:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41564000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3658:x1868674269158400:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1782107179.460155:0:17563:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1782107179.460184:0:17563:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41564000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3658:x1868674269158400:12345-0@lo:13:osp-pre-1-1.0 Request processed in 39us (275us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107179.460189:0:17563:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00010000:00080000:1.0:1782107179.460232:0:3661:0:(ldlm_request.c:1663:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffffa08b4155fb80 x1868674269158272/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1782107195 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1782107179.460247:0:3661:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3661:1868674269158272:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1782107179.460312:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00080000:2.0:1782107179.460332:0:3658:0:(ldlm_request.c:1663:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffffa08b41565f80 x1868674269158400/t0(0) o13->lustre-OST0001-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 0 dl 1782107195 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1782107179.460344:0:3658:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b41565f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3658:1868674269158400:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1782107179.460835:0:3659:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7bbfb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:43809427-66c4-4a66-b088-77a1495576b0:3659:1868674269158528:0@lo:400:kworker.0 00000100:00100000:2.0:1782107179.460869:0:3659:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782107179.460923:0:6511:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269158528 00000100:00100000:3.0:1782107179.460929:0:6511:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269158528 00000100:00100000:3.0:1782107179.460967:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1782107179.460976:0:6511:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c42d5fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:43809427-66c4-4a66-b088-77a1495576b0+23:3659:x1868674269158528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.466351:0:6511:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c42d5fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:43809427-66c4-4a66-b088-77a1495576b0+23:3659:x1868674269158528:12345-0@lo:400:kworker.0 Request processed in 5367us (5470us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107179.466367:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1782107179.466463:0:3660:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3660:1868674269158656:0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.466566:0:3660:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782107179.466598:0:3660:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1782107179.466605:0:3660:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7fc20e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269158784:0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.467905:0:3660:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782107179.467970:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269158784 00000100:00100000:3.0:1782107179.467978:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269158784 00000100:00100000:0.0:1782107179.470017:0:18105:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269158656 00000100:00100000:0.0:1782107179.470028:0:18105:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269158656 00000100:00100000:0.0:1782107179.470368:0:18105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 623 00000100:00100000:0.0:1782107179.470379:0:18105:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3660:x1868674269158656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782107179.470556:0:18105:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3660:x1868674269158656:12345-0@lo:400:kworker.0 Request processed in 178us (3989us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107179.470563:0:18105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 623 00000100:00100000:0.0:1782107179.470627:0:3660:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3660:1868674269158656:0@lo:400:kworker.0 00000100:00100000:0.0:1782107179.470926:0:3658:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7fc20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3658:1868674269158912:0@lo:400:kworker.0 00000100:00100000:0.0:1782107179.470967:0:3658:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107179.471019:0:6516:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269158912 00000100:00100000:0.0:1782107179.471023:0:6516:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269158912 00000100:00100000:0.0:1782107179.471297:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 624 00000100:00100000:0.0:1782107179.471305:0:6516:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3658:x1868674269158912:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782107179.471353:0:6516:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3658:x1868674269158912:12345-0@lo:400:kworker.0 Request processed in 50us (387us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107179.471376:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 624 00000100:00100000:0.0:1782107179.471434:0:3658:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7fc20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3658:1868674269158912:0@lo:400:kworker.0 00000100:00100000:2.0:1782107179.471519:0:3659:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7bbfb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:43809427-66c4-4a66-b088-77a1495576b0:3659:1868674269158528:0@lo:400:kworker.0 00000100:00100000:1.0:1782107179.471690:0:3658:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1782107179.471701:0:3658:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7fc20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3658:1868674269159040:0@lo:400:kworker.0 00000100:00100000:1.0:1782107179.471734:0:3658:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782107179.477348:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269159040 00000100:00100000:3.0:1782107179.477562:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269159040 00000100:00100000:2.0:1782107179.477596:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1782107179.477604:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:2.0:1782107179.477611:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:3660:x1868674269158784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.477614:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3658:x1868674269159040:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782107179.481635:0:3661:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1782107179.481670:0:3661:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3661:1868674269159168:0@lo:400:kworker.0 00000100:00100000:1.0:1782107179.481694:0:3660:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782107179.481789:0:3660:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3660:1868674269159296:0@lo:400:kworker.0 00000100:00100000:2.0:1782107179.483514:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:3660:x1868674269158784:12345-0@lo:400:kworker.0 Request processed in 5900us (15603us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107179.483528:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1782107179.483584:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3658:x1868674269159040:12345-0@lo:400:kworker.0 Request processed in 5971us (11846us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107179.483594:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:0.0:1782107179.483833:0:3661:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107179.483846:0:3658:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7ba50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3658:1868674269159424:0@lo:400:kworker.0 00000100:00100000:2.0:1782107179.484466:0:3659:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7ba50a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3659:1868674269159552:0@lo:400:kworker.0 00000100:00100000:2.0:1782107179.484499:0:3659:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107179.489451:0:3660:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782107179.490047:0:7836:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269159168 00000100:00100000:3.0:1782107179.490059:0:7836:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269159168 00000100:00080000:2.0:1782107179.490126:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1782107179.494349:0:7836:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269159552 00000100:00100000:3.0:1782107179.494366:0:7836:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269159552 00000100:00100000:3.0:1782107179.494474:0:7836:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269159296 00000100:00100000:3.0:1782107179.494477:0:7836:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269159296 00000100:00100000:3.0:1782107179.494511:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:3.0:1782107179.494522:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3661:x1868674269159168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.494621:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3661:x1868674269159168:12345-0@lo:400:kworker.0 Request processed in 102us (10788us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107179.494628:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:3.0:1782107179.494646:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:3.0:1782107179.494652:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41565f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3659:x1868674269159552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.494686:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41565f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3659:x1868674269159552:12345-0@lo:400:kworker.0 Request processed in 35us (10187us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107179.494692:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:3.0:1782107179.494704:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 627 00000100:00100000:3.0:1782107179.494909:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3660:x1868674269159296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.494966:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3660:x1868674269159296:12345-0@lo:400:kworker.0 Request processed in 61us (5524us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107179.494972:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 627 00000100:00080000:3.0:1782107179.495237:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1782107179.495288:0:3659:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7ba50a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3659:1868674269159552:0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.495355:0:3661:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3661:1868674269159168:0@lo:400:kworker.0 00000100:00100000:1.0:1782107179.501591:0:3658:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107179.501755:0:3658:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7fc20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3658:1868674269159040:0@lo:400:kworker.0 00000100:00100000:0.0:1782107179.501904:0:3660:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3660:1868674269159296:0@lo:400:kworker.0 00000100:00100000:0.0:1782107179.501936:0:3660:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7fc20e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269158784:0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.504288:0:7836:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269159424 00000100:00100000:3.0:1782107179.504302:0:7836:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269159424 00000100:00100000:3.0:1782107179.504484:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 628 00000100:00100000:3.0:1782107179.504494:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3658:x1868674269159424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782107179.504590:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3658:x1868674269159424:12345-0@lo:400:kworker.0 Request processed in 98us (3008us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107179.504597:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 628 00000100:00080000:2.0:1782107179.504652:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1782107179.504746:0:3658:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7ba50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3658:1868674269159424:0@lo:400:kworker.0 00010000:00010000:2.0:1782107180.038330:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1782107180.172603:0:22093:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-ec test 23c: fallocate on EC file ============== 01:46:18 (1782107178) 00010000:00010000:2.0:1782107181.059678:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782107181.107904:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107181.109098:0:6516:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246978560 00000100:00100000:3.0:1782107181.109110:0:6516:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246978560 00000100:00100000:3.0:1782107181.109199:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 185 00000100:00100000:3.0:1782107181.109212:0:6516:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41565880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+5:34400:x1868674246978560:12345-192.168.203.43@tcp:41:stat.0 00000004:00000024:3.0:1782107181.109266:0:6516:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 7847059456 avail 7384014848 00000100:00100000:3.0:1782107181.109343:0:6516:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41565880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+5:34400:x1868674246978560:12345-192.168.203.43@tcp:41:stat.0 Request processed in 134us (1448us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107181.109351:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 185 00000100:00080000:2.0:1782107181.109472:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1782107181.163878:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107181.164158:0:18105:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246978688 00000100:00100000:2.0:1782107181.164170:0:18105:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246978688 00000100:00100000:2.0:1782107181.164251:0:18105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 186 00000100:00100000:2.0:1782107181.164261:0:18105:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+5:34404:x1868674246978688:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1782107181.164307:0:18105:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1782107181.164327:0:18105:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419e00/0x7e3d6a8eb3a12f01 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e54 expref: -99 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.164390:0:18105:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.164449:0:18105:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.164460:0:18105:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1782107181.164507:0:18105:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1782107181.164511:0:18105:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1782107181.164530:0:18105:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.164552:0:18105:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419e00/0x7e3d6a8eb3a12f01 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 7 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.164568:0:18105:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 6 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.164577:0:18105:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:2.0:1782107181.164580:0:18105:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1782107181.164584:0:18105:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b43418200, rc 0) 00000100:00100000:2.0:1782107181.164644:0:18105:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:34404:x1868674246978688:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 384us (779us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.164651:0:18105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 186 00000100:00100000:1.0:1782107181.168542:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1782107181.168709:0:7836:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246978816 00000100:00100000:3.0:1782107181.168725:0:7836:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246978816 00000100:00100000:3.0:1782107181.168819:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 187 00000100:00100000:3.0:1782107181.168834:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:34404:x1868674246978816:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1782107181.169257:0:7836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1782107181.169284:0:7836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f0f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e5b expref: -99 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.169375:0:7836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.169394:0:7836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.169467:0:7836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.169487:0:7836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170673:0:7836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170695:0:7836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170754:0:7836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170762:0:7836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170769:0:7836:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170798:0:7836:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42907e00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:3.0:1782107181.170802:0:7836:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170810:0:7836:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170823:0:7836:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:3.0:1782107181.170826:0:7836:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107181.170833:0:7836:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170840:0:7836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907e00/0x7e3d6a8eb3a12f16 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170856:0:7836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170862:0:7836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170868:0:7836:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170873:0:7836:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42907a00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:3.0:1782107181.170876:0:7836:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170882:0:7836:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170900:0:7836:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb790e34b (ffffa08c7a609700) 00010000:00010000:3.0:1782107181.170903:0:7836:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107181.170906:0:7836:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170912:0:7836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42907a00/0x7e3d6a8eb3a12f1d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170963:0:7836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f0f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e5b expref: 7 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.170987:0:7836:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:3.0:1782107181.170989:0:7836:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107181.170994:0:7836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f0f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e5b expref: 7 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.171758:0:7836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b42906600, rc 0) 00000100:00100000:3.0:1782107181.171870:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:34404:x1868674246978816:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3035us (3332us total) trans 0 rc 301/301 00000100:00100000:3.0:1782107181.171881:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 187 00000100:00100000:2.0:1782107181.192722:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1782107181.192894:0:6518:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246979072 00000100:00100000:1.0:1782107181.192906:0:6518:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246979072 00000100:00100000:1.0:1782107181.192994:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 188 00000100:00100000:1.0:1782107181.193016:0:6518:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41565500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+12:34404:x1868674246979072:12345-192.168.203.43@tcp:36:lfs.0 00000100:00100000:1.0:1782107181.193168:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269159680:0@lo:1000:mdt00_002.0 00000100:00080000:2.0:1782107181.193741:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1782107181.193789:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.193876:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c1f00 going to sleep for 11 seconds 00000100:00100000:0.0:1782107181.193952:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269159680 00000100:00100000:0.0:1782107181.193959:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269159680 00000100:00100000:0.0:1782107181.194037:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:0.0:1782107181.194046:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269159680:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1782107181.194295:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269159680:12345-0@lo:1000:mdt00_002.0 Request processed in 250us (507us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107181.194303:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:1.0:1782107181.194442:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269159680:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.194505:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269159808:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.194529:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.194565:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c1f00 going to sleep for 11 seconds 00000100:00100000:2.0:1782107181.194696:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269159808 00000100:00100000:2.0:1782107181.194701:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269159808 00000100:00100000:2.0:1782107181.194734:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1782107181.194741:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269159808:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1782107181.194832:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269159808:12345-0@lo:1000:mdt00_002.0 Request processed in 92us (301us total) trans 0 rc -2/-2 00000100:00100000:2.0:1782107181.194839:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:1.0:1782107181.194908:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269159808:0@lo:1000:mdt00_002.0 00010000:00010000:1.0:1782107181.194953:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.194964:0:6518:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.194973:0:6518:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107181.194985:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269159936:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1782107181.195007:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.195028:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c1540 going to sleep for 11 seconds 00000100:00100000:0.0:1782107181.195179:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269159936 00000100:00100000:0.0:1782107181.195194:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269159936 00000100:00100000:0.0:1782107181.195222:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:0.0:1782107181.195229:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6518:x1868674269159936:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1782107181.195239:0:7835:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1782107181.195252:0:7835:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12f2b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7e3d6a8eb3a12f24 expref: -99 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.195290:0:7835:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 5 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.195317:0:7835:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12f2b lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f24 expref: 10 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.195350:0:7835:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 5 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.195363:0:7835:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 5 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107181.195374:0:7835:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155ea00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:7835:1868674269160064:192.168.203.43@tcp:104: 00000100:00100000:0.0:1782107181.195463:0:7835:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08c81af0c00 going to sleep for 11 seconds 00000100:00100000:0.0:1782107181.203589:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1782107181.203770:0:9470:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246979200 00000100:00100000:1.0:1782107181.203779:0:9470:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246979200 00010000:00010000:1.0:1782107181.203817:0:9470:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 6 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107181.203867:0:9470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 72 00000100:00100000:1.0:1782107181.203877:0:9470:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:2414:x1868674246979200:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1782107181.203891:0:9470:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782107181.203897:0:9470:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 6 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.203938:0:9470:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 6 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.203960:0:9470:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:1.0:1782107181.203966:0:9470:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12f2b lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f24 expref: 10 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.203978:0:9470:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12f2b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f24 expref: 10 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.203987:0:9470:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12f2b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f24 expref: 10 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.204021:0:9470:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12f2b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f24 expref: 10 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107181.204034:0:9470:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f32a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:9470:1868674269160192:0@lo:105: 00000100:00100000:1.0:1782107181.204076:0:9470:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.204107:0:9470:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c1f00 going to sleep for 11 seconds 00000100:00100000:2.0:1782107181.204175:0:6499:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269160192 00000100:00100000:2.0:1782107181.204180:0:6499:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269160192 00000100:00100000:2.0:1782107181.204226:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 51 00000100:00100000:2.0:1782107181.204233:0:6499:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f32d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9470:x1868674269160192:12345-0@lo:105: 00010000:00010000:2.0:1782107181.204243:0:6499:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.204253:0:6499:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.204260:0:6499:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.204267:0:6499:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.204273:0:6499:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.204281:0:6499:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.204301:0:6499:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa08c412eb200) 00000100:00100000:2.0:1782107181.204343:0:6499:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f32d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9470:x1868674269160192:12345-0@lo:105: Request processed in 110us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.204349:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 51 00000100:00100000:2.0:1782107181.204525:0:9470:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f32a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:9470:1868674269160192:0@lo:105: 00010000:00010000:2.0:1782107181.204543:0:9470:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1782107181.204578:0:9470:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:2414:x1868674246979200:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 703us (998us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.204585:0:9470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 72 00010000:00010000:3.0:1782107181.206080:0:7835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418200/0x7e3d6a8eb3a12f08 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e54 expref: 5 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1782107181.206103:0:7835:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155ea00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:7835:1868674269160064:192.168.203.43@tcp:104: 00010000:00010000:3.0:1782107181.206130:0:7835:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12f2b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f24 expref: 10 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.206160:0:7835:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:3.0:1782107181.206162:0:7835:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107181.206166:0:7835:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c40030a00, rc 0) 00000100:00100000:2.0:1782107181.206384:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6518:x1868674269159936:12345-0@lo:101:mdt00_002.0 Request processed in 11154us (11374us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.206421:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1782107181.206762:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269159936:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1782107181.206778:0:6518:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1782107181.206797:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1782107181.207145:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.207168:0:6518:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.207178:0:6518:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107181.207206:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160320:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1782107181.207239:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.207309:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c1540 going to sleep for 11 seconds 00000100:00100000:0.0:1782107181.207356:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269160320 00000100:00100000:0.0:1782107181.207361:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269160320 00000100:00100000:0.0:1782107181.207507:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:0.0:1782107181.207517:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6518:x1868674269160320:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1782107181.207531:0:6522:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1782107181.207557:0:6522:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12f39 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7e3d6a8eb3a12f32 expref: -99 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.207579:0:6522:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12f39 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f32 expref: 11 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.207592:0:6522:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12f39 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f32 expref: 11 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.207610:0:6522:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb790e34b (ffffa08b42efa200) 00010000:00010000:0.0:1782107181.207613:0:6522:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1782107181.207616:0:6522:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c40031000, rc 0) 00000100:00100000:0.0:1782107181.207647:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160320:12345-0@lo:101:mdt00_002.0 Request processed in 133us (407us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107181.207653:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:1.0:1782107181.207709:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160320:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1782107181.207720:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.207729:0:6518:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1782107181.207740:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1782107181.207770:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160448:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.207796:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.208073:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1782107181.208142:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269160448 00000100:00100000:2.0:1782107181.208148:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269160448 00000100:00100000:2.0:1782107181.208172:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:2.0:1782107181.208179:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160448:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1782107181.208293:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160448:12345-0@lo:1000:mdt00_002.0 Request processed in 114us (495us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.208299:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1782107181.208398:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160448:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.208452:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160576:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.208477:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.208665:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c13c0 going to sleep for 11 seconds 00000100:00100000:3.0:1782107181.208726:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269160576 00000100:00100000:3.0:1782107181.208731:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269160576 00000100:00100000:2.0:1782107181.208790:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:2.0:1782107181.208798:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160576:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1782107181.208871:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160576:12345-0@lo:1000:mdt00_002.0 Request processed in 74us (392us total) trans 0 rc -2/-2 00000100:00100000:2.0:1782107181.208877:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1782107181.209035:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160576:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.209105:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160704:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.209136:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.209227:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1782107181.209264:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269160704 00000100:00100000:0.0:1782107181.209268:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269160704 00000100:00100000:0.0:1782107181.209298:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:0.0:1782107181.209305:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160704:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1782107181.209476:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160704:12345-0@lo:1000:mdt00_002.0 Request processed in 171us (340us total) trans 0 rc -2/-2 00000100:00100000:0.0:1782107181.209484:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1782107181.209887:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160704:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.210055:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160832:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.210081:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.210137:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1782107181.210211:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269160832 00000100:00100000:2.0:1782107181.210215:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269160832 00000100:00100000:2.0:1782107181.210277:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1782107181.210285:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160832:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1782107181.210386:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160832:12345-0@lo:1000:mdt00_002.0 Request processed in 102us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.210393:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:1.0:1782107181.210743:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160832:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.210785:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160960:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.210809:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.210832:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1782107181.210890:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269160960 00000100:00100000:0.0:1782107181.210895:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269160960 00000100:00100000:0.0:1782107181.210936:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:0.0:1782107181.210943:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160960:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1782107181.211013:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269160960:12345-0@lo:1000:mdt00_002.0 Request processed in 70us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107181.211020:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1782107181.211164:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269160960:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.211212:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269161088:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1782107181.211234:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.211257:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1782107181.211305:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269161088 00000100:00100000:0.0:1782107181.211309:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269161088 00000100:00100000:0.0:1782107181.211337:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1782107181.211343:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269161088:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1782107181.211392:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6518:x1868674269161088:12345-0@lo:1000:mdt00_002.0 Request processed in 50us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107181.211431:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:1.0:1782107181.211570:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269161088:0@lo:1000:mdt00_002.0 00000040:00080000:1.0:1782107181.212712:0:6518:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].5, 2152 off41096 00000040:00080000:1.0:1782107181.212792:0:6518:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].5, 2352 off41856 00000004:00080000:1.0:1782107181.212798:0:6518:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa08b434c1540 with version 9 00000004:00080000:2.0:1782107181.212886:0:7445:0:(osp_trans.c:1330:osp_get_next_request()) ou ffffa08c453a7840 version 9 rpc_version 9 00000020:00100000:2.0:1782107181.213033:0:7445:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1782107181.213044:0:7445:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa08b41566680 x1868674269161344/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782107181.213064:0:7445:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b41566680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269161344:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1782107181.213116:0:7445:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107181.213179:0:7445:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9240 going to sleep for 11 seconds 00000100:00100000:3.0:1782107181.213230:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269161344 00000100:00100000:3.0:1782107181.213236:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269161344 00000100:00100000:3.0:1782107181.213321:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:3.0:1782107181.213330:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41567100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7445:x1868674269161344:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1782107181.213899:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41567100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7445:x1868674269161344:12345-0@lo:1000:osp_up0-1.0 Request processed in 569us (781us total) trans 4294967363 rc 0/0 00000100:00100000:3.0:1782107181.213921:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:2.0:1782107181.214051:0:7445:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b41566680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269161344:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1782107181.215009:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.215026:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.215038:0:6518:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.215072:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.215080:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.215086:0:6518:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.215116:0:6518:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6518 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.215122:0:6518:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.215130:0:6518:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107181.215148:0:6518:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269161472:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1782107181.215199:0:6518:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.215226:0:6518:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b434c1f00 going to sleep for 11 seconds 00000100:00100000:3.0:1782107181.215354:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269161472 00000100:00100000:3.0:1782107181.215361:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269161472 00000100:00100000:3.0:1782107181.215441:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1782107181.215449:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6518:x1868674269161472:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1782107181.215471:0:7835:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1782107181.215486:0:7835:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f55 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7e3d6a8eb3a12f4e expref: -99 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.215508:0:7835:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f55 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f4e expref: 13 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.215519:0:7835:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f55 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f4e expref: 13 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.215538:0:7835:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffffa08c7a609700) 00010000:00010000:3.0:1782107181.215541:0:7835:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107181.215545:0:7835:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b42906600, rc 0) 00000100:00100000:3.0:1782107181.215582:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:6518:x1868674269161472:12345-0@lo:101:mdt00_002.0 Request processed in 134us (382us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107181.215588:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:2.0:1782107181.215699:0:6518:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42f30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6518:1868674269161472:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1782107181.215714:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215723:0:6518:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1782107181.215742:0:6518:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000003456ffc3 reply state 000000002eab7a93 transno 4294967317 ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215753:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215771:0:6518:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0x0 (ffffa08c469af300) 00010000:00010000:2.0:1782107181.215773:0:6518:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1782107181.215776:0:6518:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1782107181.215783:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215791:0:6518:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1782107181.215800:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000401:0x6:0x0] 00000004:00010000:2.0:1782107181.215806:0:6518:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000003456ffc3 reply state 000000002eab7a93 transno 4294967317 ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215814:0:6518:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215820:0:6518:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5:0x0].0x0 (ffffa08c469af000) 00010000:00010000:2.0:1782107181.215821:0:6518:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1782107181.215823:0:6518:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1782107181.215828:0:6518:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x5:0x0]: returning size 4096 00000004:00080000:2.0:1782107181.215834:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1782107181.215843:0:6518:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215850:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215858:0:6518:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215929:0:6518:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1782107181.215935:0:6518:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1782107181.215940:0:6518:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.215948:0:6518:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107181.216024:0:6518:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41565500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+14:34404:x1868674246979072:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 23019us (23315us total) trans 4294967317 rc 0/0 00000100:00100000:2.0:1782107181.216032:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 188 00010000:00010000:3.0:1782107181.217208:0:6505:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1782107181.217218:0:6505:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.217229:0:6505:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c412eb200 already unused, calling callback (ffffffffc1a15ee0) 00010000:00010000:3.0:1782107181.217235:0:6505:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1782107181.217244:0:6505:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.218001:0:6505:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.218012:0:6505:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1782107181.218042:0:6505:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.218050:0:6505:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eb200/0x7e3d6a8eb3a12f24 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12f2b expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.218064:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782107181.218066:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782107181.218068:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782107181.218172:0:3660:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b41566d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269161600:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1782107181.218241:0:3660:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107181.219106:0:6503:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269161600 00000100:00100000:1.0:1782107181.219114:0:6503:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269161600 00000100:00100000:1.0:1782107181.219172:0:6503:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14 00000100:00100000:1.0:1782107181.219181:0:6503:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3660:x1868674269161600:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1782107181.219207:0:6503:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782107181.219227:0:6503:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12f2b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f24 expref: 13 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.219243:0:6503:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:1.0:1782107181.219259:0:6503:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1782107181.221140:0:3660:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b41566d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269161600:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782107181.221211:0:6503:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3660:x1868674269161600:12345-0@lo:103:ldlm_bl.0 Request processed in 2030us (2969us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107181.221223:0:6503:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14 00000100:00100000:0.0:1782107181.243991:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107181.244165:0:18105:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246979328 00000100:00100000:2.0:1782107181.244175:0:18105:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246979328 00000100:00100000:2.0:1782107181.244243:0:18105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:2.0:1782107181.244252:0:18105:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+5:34407:x1868674246979328:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1782107181.244291:0:18105:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1782107181.244309:0:18105:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419000/0x7e3d6a8eb3a12f5c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e62 expref: -99 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.244373:0:18105:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418800/0x7e3d6a8eb3a12f63 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.244388:0:18105:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418800/0x7e3d6a8eb3a12f63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.244425:0:18105:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418800/0x7e3d6a8eb3a12f63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1782107181.244462:0:18105:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1782107181.244465:0:18105:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1782107181.244483:0:18105:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418800/0x7e3d6a8eb3a12f63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.244513:0:18105:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419000/0x7e3d6a8eb3a12f5c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e62 expref: 7 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.244530:0:18105:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418800/0x7e3d6a8eb3a12f63 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e62 expref: 6 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.244547:0:18105:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:2.0:1782107181.244550:0:18105:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1782107181.244554:0:18105:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b43418800, rc 0) 00000100:00100000:2.0:1782107181.244612:0:18105:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:34407:x1868674246979328:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 361us (630us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.244619:0:18105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:1.0:1782107181.252761:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107181.252901:0:6517:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246979456 00000100:00100000:2.0:1782107181.252911:0:6517:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246979456 00000100:00100000:2.0:1782107181.252965:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00100000:2.0:1782107181.252975:0:6517:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42f30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2eadce33-9ab8-4caf-b349-1b7b3db092bc+6:34407:x1868674246979456:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1782107181.252996:0:6517:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1782107181.253016:0:6517:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12f6a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e69 expref: -99 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.253081:0:6517:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.253096:0:6517:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.253108:0:6517:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.253118:0:6517:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.253128:0:6517:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12f39 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f32 expref: 11 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.253136:0:6517:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.253167:0:6517:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12f39 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7e3d6a8eb3a12f32 expref: 11 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107181.253193:0:6517:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b41567b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6517:1868674269161728:0@lo:104: 00000100:00100000:2.0:1782107181.253254:0:6517:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107181.253316:0:6517:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9240 going to sleep for 11 seconds 00000100:00100000:3.0:1782107181.253666:0:6499:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269161728 00000100:00100000:3.0:1782107181.253672:0:6499:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269161728 00000100:00100000:3.0:1782107181.253709:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:3.0:1782107181.253716:0:6499:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41567480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6517:x1868674269161728:12345-0@lo:104: 00010000:00010000:3.0:1782107181.253728:0:6499:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.253788:0:6499:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1782107181.253793:0:6499:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41567480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6517:x1868674269161728:12345-0@lo:104: Request processed in 79us (540us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107181.253809:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00010000:00010000:0.0:1782107181.253881:0:6504:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1782107181.253887:0:6504:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.254214:0:6504:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c412ebe00 already unused, calling callback (ffffffffc1a15ee0) 00010000:00010000:0.0:1782107181.254219:0:6504:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1782107181.254227:0:6504:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.254255:0:6504:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.254261:0:6504:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1782107181.254285:0:6504:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.254293:0:6504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412ebe00/0x7e3d6a8eb3a12f32 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7e3d6a8eb3a12f39 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.254314:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782107181.254322:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782107181.254324:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782107181.254333:0:3660:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1782107181.254344:0:3660:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269161856:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1782107181.254379:0:3660:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107181.254494:0:6517:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b41567b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6517:1868674269161728:0@lo:104: 00010000:00010000:2.0:1782107181.254514:0:6517:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107181.255102:0:6502:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269161856 00000100:00100000:0.0:1782107181.255109:0:6502:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269161856 00010000:00010000:0.0:1782107181.255131:0:6502:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12f39 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7e3d6a8eb3a12f32 expref: 12 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107181.255156:0:6502:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 73 00000100:00100000:0.0:1782107181.255162:0:6502:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3660:x1868674269161856:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1782107181.255173:0:6502:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1782107181.262429:0:6495:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967363 is committed 00010000:00010000:0.0:1782107181.262751:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12f39 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7e3d6a8eb3a12f32 expref: 9 pid: 6522 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.262772:0:6502:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb790e34b (ffffa08b42efa200) 00010000:00010000:0.0:1782107181.262777:0:6502:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.262787:0:6502:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.262796:0:6502:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.262882:0:6502:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1782107181.263492:0:6502:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3660:x1868674269161856:12345-0@lo:103:ldlm_bl.0 Request processed in 8325us (9108us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107181.263504:0:6502:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 73 00000100:00100000:3.0:1782107181.263643:0:3660:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269161856:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1782107181.264079:0:6517:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.264091:0:6517:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107181.264219:0:6517:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b41567b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6517:1868674269161984:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1782107181.264271:0:6517:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107181.264347:0:6517:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9000 going to sleep for 11 seconds 00000100:00100000:3.0:1782107181.264390:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269161984 00000100:00100000:3.0:1782107181.264510:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269161984 00000100:00100000:3.0:1782107181.264599:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:3.0:1782107181.264610:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41566d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6517:x1868674269161984:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1782107181.264692:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41566d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6517:x1868674269161984:12345-0@lo:1000:mdt00_001.0 Request processed in 85us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107181.264699:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:1.0:1782107181.264759:0:6517:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b41567b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6517:1868674269161984:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1782107181.264806:0:6517:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eae00/0x7e3d6a8eb3a12f7f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264820:0:6517:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eae00/0x7e3d6a8eb3a12f7f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264831:0:6517:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eae00/0x7e3d6a8eb3a12f7f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1782107181.264854:0:6517:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eae00/0x7e3d6a8eb3a12f7f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264879:0:6517:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264888:0:6517:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264894:0:6517:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264911:0:6517:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264917:0:6517:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b43418a00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:1.0:1782107181.264921:0:6517:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264939:0:6517:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264960:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa08c43d2b100) 00010000:00010000:1.0:1782107181.264963:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1782107181.264968:0:6517:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264974:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43418a00/0x7e3d6a8eb3a12f71 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264986:0:6517:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264993:0:6517:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.264999:0:6517:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.265003:0:6517:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b43419800 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:1.0:1782107181.265006:0:6517:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.265011:0:6517:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.265017:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb790e34b (ffffa08b42efa200) 00010000:00010000:1.0:1782107181.265020:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1782107181.265022:0:6517:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.265037:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419800/0x7e3d6a8eb3a12f78 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb790e34b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.265076:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12f6a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e69 expref: 8 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.265089:0:6517:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08c412eae00/0x7e3d6a8eb3a12f7f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e69 expref: 7 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.265097:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5:0x0].0x0 (ffffa08c469afa00) 00010000:00010000:1.0:1782107181.265099:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1782107181.265102:0:6517:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c412eae00, rc 0) 00000100:00100000:1.0:1782107181.265157:0:6517:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42f30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2eadce33-9ab8-4caf-b349-1b7b3db092bc+7:34407:x1868674246979456:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 12184us (12404us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107181.265164:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00100000:3.0:1782107181.268808:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1782107181.268911:0:6516:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246979584 00000100:00100000:0.0:1782107181.268921:0:6516:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246979584 00000100:00100000:0.0:1782107181.268987:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 191 00000100:00100000:0.0:1782107181.268998:0:6516:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+14:34407:x1868674246979584:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1782107181.269022:0:6516:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1782107181.269046:0:6516:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12f86 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e70 expref: -99 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.269107:0:6516:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031c00/0x7e3d6a8eb3a12f8d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6516 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.269122:0:6516:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031c00/0x7e3d6a8eb3a12f8d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x5:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.269132:0:6516:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031c00/0x7e3d6a8eb3a12f8d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x5:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1782107181.269235:0:6516:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1782107181.269240:0:6516:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1782107181.269278:0:6516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12f86 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x5:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e70 expref: 16 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.269296:0:6516:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031c00/0x7e3d6a8eb3a12f8d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x5:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e70 expref: 15 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.269305:0:6516:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5:0x0].0x0 (ffffa08c469af000) 00010000:00010000:0.0:1782107181.269308:0:6516:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1782107181.269312:0:6516:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c40031c00, rc 0) 00000100:00100000:0.0:1782107181.269368:0:6516:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+15:34407:x1868674246979584:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 372us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107181.269390:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 191 00000100:00100000:0.0:1782107181.272138:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107181.272225:0:18105:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246979712 00000100:00100000:2.0:1782107181.272235:0:18105:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246979712 00000100:00100000:2.0:1782107181.272294:0:18105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 192 00000100:00100000:2.0:1782107181.272304:0:18105:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b4155fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+15:34407:x1868674246979712:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1782107181.272322:0:18105:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1782107181.272340:0:18105:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419000/0x7e3d6a8eb3a12f94 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e77 expref: -99 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.272428:0:18105:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418e00/0x7e3d6a8eb3a12f9b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.272441:0:18105:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418e00/0x7e3d6a8eb3a12f9b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.272471:0:18105:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418e00/0x7e3d6a8eb3a12f9b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1782107181.272518:0:18105:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000400:0x6:0x0]: returning size 4096 00000004:00200000:2.0:1782107181.272523:0:18105:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1782107181.272556:0:18105:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419000/0x7e3d6a8eb3a12f94 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e77 expref: 17 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.272572:0:18105:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418e00/0x7e3d6a8eb3a12f9b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e77 expref: 16 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.272580:0:18105:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0x0 (ffffa08c469af300) 00010000:00010000:2.0:1782107181.272583:0:18105:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1782107181.272586:0:18105:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b43418e00, rc 0) 00000100:00100000:2.0:1782107181.272639:0:18105:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b4155fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+16:34407:x1868674246979712:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 336us (508us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.272646:0:18105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 192 00000100:00100000:1.0:1782107181.275305:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1782107181.275364:0:7836:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246979840 00000100:00100000:0.0:1782107181.275373:0:7836:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246979840 00000100:00100000:0.0:1782107181.275455:0:7836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:0.0:1782107181.275466:0:7836:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+7:34407:x1868674246979840:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1782107181.275484:0:7836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1782107181.275502:0:7836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fa2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e7e expref: -99 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.275557:0:7836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.275571:0:7836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f55 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12f4e expref: 7 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.275580:0:7836:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.275613:0:7836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f55 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7e3d6a8eb3a12f4e expref: 7 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782107181.275627:0:7836:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b4155c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7836:1868674269162112:0@lo:104: 00000100:00100000:0.0:1782107181.275668:0:7836:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107181.275694:0:7836:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08c81af0cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1782107181.275830:0:6499:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269162112 00000100:00100000:3.0:1782107181.275835:0:6499:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269162112 00000100:00100000:3.0:1782107181.275870:0:6499:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:3.0:1782107181.275877:0:6499:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7bbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7836:x1868674269162112:12345-0@lo:104: 00010000:00010000:3.0:1782107181.275889:0:6499:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107181.276053:0:6499:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1782107181.276059:0:6499:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7bbfb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7836:x1868674269162112:12345-0@lo:104: Request processed in 182us (390us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107181.276066:0:6499:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00010000:00010000:2.0:1782107181.276121:0:6505:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1782107181.276128:0:6505:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.276136:0:6505:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c412eac00 already unused, calling callback (ffffffffc1a15ee0) 00010000:00010000:2.0:1782107181.276141:0:6505:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1782107181.276149:0:6505:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.276201:0:6505:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.276208:0:6505:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1782107181.276236:0:6505:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 1/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.276243:0:6505:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08c412eac00/0x7e3d6a8eb3a12f4e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7e3d6a8eb3a12f55 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.276262:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782107181.276264:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782107181.276266:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782107181.276303:0:3658:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b41567b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3658:1868674269162240:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782107181.276335:0:3658:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107181.276425:0:9469:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269162240 00000100:00100000:2.0:1782107181.276430:0:9469:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269162240 00010000:00010000:2.0:1782107181.276449:0:9469:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f55 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7e3d6a8eb3a12f4e expref: 8 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107181.276472:0:9469:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 74 00000100:00100000:2.0:1782107181.276478:0:9469:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7fc20a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3658:x1868674269162240:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1782107181.276488:0:9469:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1782107181.276848:0:7836:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b4155c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7836:1868674269162112:0@lo:104: 00010000:00010000:0.0:1782107181.276867:0:7836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.277207:0:9469:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b42906600/0x7e3d6a8eb3a12f55 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7e3d6a8eb3a12f4e expref: 8 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.277221:0:9469:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffffa08c7a609700) 00010000:00010000:2.0:1782107181.277225:0:9469:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.277243:0:9469:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.277250:0:9469:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.277276:0:9469:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1782107181.277321:0:9469:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc20a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3658:x1868674269162240:12345-0@lo:103:ldlm_bl.0 Request processed in 843us (985us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.277329:0:9469:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 74 00000100:00100000:1.0:1782107181.277389:0:3658:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b41567b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3658:1868674269162240:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1782107181.277618:0:7836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.277626:0:7836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7836 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1782107181.277675:0:7836:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000401:0x6:0x0]: returning size 4096 00000004:00200000:0.0:1782107181.277680:0:7836:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1782107181.277721:0:7836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fa2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e7e expref: 9 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.277736:0:7836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa08c40031800/0x7e3d6a8eb3a12fa9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e7e expref: 8 pid: 7836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.277744:0:7836:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffffa08c7a609700) 00010000:00010000:0.0:1782107181.277746:0:7836:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1782107181.277749:0:7836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c40031800, rc 0) 00000100:00100000:0.0:1782107181.277799:0:7836:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7fc20e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2eadce33-9ab8-4caf-b349-1b7b3db092bc+8:34407:x1868674246979840:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2335us (2503us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107181.277806:0:7836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:3.0:1782107181.281028:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1782107181.281168:0:6516:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246979968 00000100:00100000:0.0:1782107181.281178:0:6516:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246979968 00000100:00100000:0.0:1782107181.281245:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:0.0:1782107181.281254:0:6516:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+16:34407:x1868674246979968:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1782107181.281273:0:6516:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1782107181.281300:0:6516:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12fb0 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e85 expref: -99 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281381:0:6516:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x6:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6516 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281395:0:6516:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281446:0:6516:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281508:0:6516:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 3/1,0 mode: --/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6516 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281516:0:6516:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281523:0:6516:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281577:0:6516:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281585:0:6516:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 2/0,0 mode: CR/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281591:0:6516:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281597:0:6516:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c40030a00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:0.0:1782107181.281601:0:6516:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281619:0:6516:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281629:0:6516:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0x0 (ffffa08c469af300) 00010000:00010000:0.0:1782107181.281632:0:6516:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1782107181.281637:0:6516:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281642:0:6516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030a00/0x7e3d6a8eb3a12fb7 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281655:0:6516:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281661:0:6516:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281667:0:6516:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281672:0:6516:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c40030600 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:0.0:1782107181.281674:0:6516:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281679:0:6516:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281685:0:6516:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0xdd5a5043 (ffffa08b42efab00) 00010000:00010000:0.0:1782107181.281687:0:6516:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1782107181.281690:0:6516:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 2/0,0 mode: --/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281704:0:6516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030600/0x7e3d6a8eb3a12fbe lrc: 0/0,0 mode: --/PR res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6516 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281737:0:6516:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12fb0 lrc: 1/0,0 mode: --/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e85 expref: 17 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281746:0:6516:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0x0 (ffffa08c469af300) 00010000:00010000:0.0:1782107181.281748:0:6516:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1782107181.281752:0:6516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40030000/0x7e3d6a8eb3a12fb0 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e85 expref: 17 pid: 6516 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.281762:0:6516:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c40030000, rc 0) 00000100:00100000:0.0:1782107181.281813:0:6516:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+16:34407:x1868674246979968:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 559us (793us total) trans 0 rc 301/301 00000100:00100000:0.0:1782107181.281820:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:3.0:1782107181.286314:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107181.286443:0:18105:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246980224 00000100:00100000:2.0:1782107181.286452:0:18105:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246980224 00000100:00100000:2.0:1782107181.286499:0:18105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:2.0:1782107181.286510:0:18105:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+16:34407:x1868674246980224:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1782107181.286529:0:18105:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1782107181.286542:0:18105:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1782107181.286570:0:18105:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418e00/0x7e3d6a8eb3a12f9b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e77 expref: 16 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.286587:0:18105:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0x0 (ffffa08c469af300) 00010000:00010000:2.0:1782107181.286592:0:18105:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1782107181.286604:0:18105:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419000/0x7e3d6a8eb3a12fc5 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e8c expref: -99 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.286698:0:18105:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 3/0,1 mode: --/CW res: [0x240000400:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.286711:0:18105:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.286733:0:18105:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.286748:0:18105:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.286755:0:18105:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.286760:0:18105:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107181.287394:0:18105:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b41567b80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18105:1868674269162368:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1782107181.287483:0:18105:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107181.287507:0:18105:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9840 going to sleep for 11 seconds 00000100:00100000:1.0:1782107181.287620:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269162368 00000100:00100000:1.0:1782107181.287625:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269162368 00000100:00100000:1.0:1782107181.287663:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1782107181.287671:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42fe4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:18105:x1868674269162368:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1782107181.287747:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42fe4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:18105:x1868674269162368:12345-0@lo:1000:mdt00_005.0 Request processed in 76us (262us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107181.287754:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:0.0:1782107181.287887:0:18105:0:(client.c:3102:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967363 gen 2 00000004:00080000:0.0:1782107181.287894:0:18105:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967361 committed transno 4294967363 00000004:00080000:0.0:1782107181.287926:0:18105:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967362 committed transno 4294967363 00000004:00080000:0.0:1782107181.287939:0:18105:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967363 committed transno 4294967363 00000100:00100000:0.0:1782107181.287966:0:18105:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b41567b80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18105:1868674269162368:0@lo:1000:mdt00_005.0 00010000:00010000:0.0:1782107181.288299:0:18105:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 3/1,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18105 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288312:0:18105:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288322:0:18105:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1782107181.288342:0:18105:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x6:0x0]: returning size 0 00010000:00010000:0.0:1782107181.288367:0:18105:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288374:0:18105:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288380:0:18105:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288386:0:18105:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b43419600 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:0.0:1782107181.288390:0:18105:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288397:0:18105:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288794:0:18105:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0x0 (ffffa08c469af300) 00010000:00010000:0.0:1782107181.288797:0:18105:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1782107181.288803:0:18105:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 2/0,0 mode: --/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288809:0:18105:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419600/0x7e3d6a8eb3a12fcc lrc: 0/0,0 mode: --/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1782107181.288823:0:18105:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000034974610 reply state 00000000d10fe5e3 transno 4294967318 ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288833:0:18105:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.288848:0:18105:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0xdd5a5043 (ffffa08c43d2b600) 00010000:00010000:0.0:1782107181.288850:0:18105:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1782107181.288853:0:18105:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1782107181.289069:0:18105:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419000/0x7e3d6a8eb3a12fc5 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 18 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.289084:0:18105:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 17 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107181.289090:0:18105:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffffa08b42efa900) 00010000:00010000:0.0:1782107181.289092:0:18105:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1782107181.289094:0:18105:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08c40031000, rc 0) 00000100:00100000:0.0:1782107181.289240:0:18105:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2eadce33-9ab8-4caf-b349-1b7b3db092bc+18:34407:x1868674246980224:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2732us (2935us total) trans 4294967318 rc 0/0 00000100:00100000:0.0:1782107181.289248:0:18105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:1.0:1782107181.293271:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107181.293362:0:6517:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246980480 00000100:00100000:2.0:1782107181.293372:0:6517:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246980480 00000100:00100000:2.0:1782107181.293584:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:2.0:1782107181.293595:0:6517:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7b7e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2eadce33-9ab8-4caf-b349-1b7b3db092bc+18:34407:x1868674246980480:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1782107181.293623:0:6517:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1782107181.293642:0:6517:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418600/0x7e3d6a8eb3a12fe8 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x518b9a3576b3e93 expref: -99 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.293746:0:6517:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,1 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6517 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.293761:0:6517:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 19 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.293786:0:6517:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 4/0,1 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.293818:0:6517:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 19 pid: 18105 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.293830:0:6517:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 19 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107181.293842:0:6517:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42fe6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6517:1868674269162496:192.168.203.43@tcp:104: 00000100:00100000:2.0:1782107181.294124:0:6517:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08b42fd9000 going to sleep for 11 seconds 00000100:00100000:3.0:1782107181.296945:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1782107181.297080:0:6503:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246980608 00000100:00100000:1.0:1782107181.297087:0:6503:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246980608 00010000:00010000:1.0:1782107181.297124:0:6503:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 20 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782107181.297173:0:6503:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 75 00000100:00100000:1.0:1782107181.297192:0:6503:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c4abb8050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+20:2414:x1868674246980608:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1782107181.297207:0:6503:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782107181.297213:0:6503:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 20 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.297225:0:6503:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 20 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.297244:0:6503:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c40031000/0x7e3d6a8eb3a12fda lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e8c expref: 20 pid: 18105 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.297259:0:6503:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffffa08b42efa900) 00010000:00010000:1.0:1782107181.297264:0:6503:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,1 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.297274:0:6503:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.297281:0:6503:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107181.297299:0:6503:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782107181.297365:0:6503:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c4abb8050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:2eadce33-9ab8-4caf-b349-1b7b3db092bc+19:2414:x1868674246980608:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 184us (496us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107181.297373:0:6503:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 75 00000100:00100000:2.0:1782107181.298754:0:6517:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42fe6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6517:1868674269162496:192.168.203.43@tcp:104: 00010000:00010000:2.0:1782107181.298782:0:6517:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.298794:0:6517:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.298799:0:6517:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1782107181.298825:0:6517:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x6:0x0]: returning size 0 00010000:00010000:2.0:1782107181.299646:0:6517:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299657:0:6517:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299686:0:6517:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299692:0:6517:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b43419c00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:2.0:1782107181.299696:0:6517:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299703:0:6517:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299717:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffffa08b42efa900) 00010000:00010000:2.0:1782107181.299721:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1782107181.299725:0:6517:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 2/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299731:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43419c00/0x7e3d6a8eb3a12fef lrc: 0/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299770:0:6517:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418600/0x7e3d6a8eb3a12fe8 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e93 expref: 18 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299779:0:6517:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffffa08b42efa900) 00010000:00010000:2.0:1782107181.299781:0:6517:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1782107181.299785:0:6517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418600/0x7e3d6a8eb3a12fe8 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0x518b9a3576b3e93 expref: 18 pid: 6517 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782107181.299798:0:6517:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa08b43418600, rc 0) 00000100:00100000:2.0:1782107181.299853:0:6517:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7b7e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2eadce33-9ab8-4caf-b349-1b7b3db092bc+17:34407:x1868674246980480:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 6261us (6589us total) trans 0 rc 301/301 00000100:00100000:2.0:1782107181.301483:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:3.0:1782107181.305316:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1782107181.305426:0:6519:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674246980864 00000100:00100000:2.0:1782107181.305438:0:6519:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674246980864 00000100:00100000:2.0:1782107181.305503:0:6519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 31 00000100:00100000:2.0:1782107181.305514:0:6519:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+17:34407:x1868674246980864:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:2.0:1782107181.305731:0:6519:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:2eadce33-9ab8-4caf-b349-1b7b3db092bc+18:34407:x1868674246980864:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 218us (421us total) trans 4294967319 rc 0/0 00000100:00100000:2.0:1782107181.305739:0:6519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 31 00000100:00100000:3.0:1782107181.508234:0:3659:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3659:1868674269162624:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1782107181.508322:0:3659:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782107181.508526:0:6521:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269162624 00000100:00100000:2.0:1782107181.508546:0:6521:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269162624 00000100:00100000:2.0:1782107181.508593:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1782107181.508601:0:6521:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3659:x1868674269162624:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1782107181.508633:0:6521:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446371328 avail 1300131840 00000100:00100000:2.0:1782107181.508676:0:6521:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3659:x1868674269162624:12345-0@lo:41:osp-pre-1-0.0 Request processed in 75us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1782107181.508683:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:3.0:1782107181.508926:0:3659:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3659:1868674269162624:0@lo:41:osp-pre-1-0.0 00010000:00010000:0.0:1782107182.090513:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1782107182.768114:0:22190:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-ec test_23c: @@@@@@ FAIL: setstripe failed 00010000:00010000:2.0:1782107183.107569:0:6505:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782107183.558448:0:7431:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7e3d6a8eb3a12ed0 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1782107183.558466:0:7431:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7e3d6a8eb3a12ed0 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107183.558498:0:7431:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12ed0 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782107183.558504:0:7431:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1782107183.558512:0:7431:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa08b43419200/0x7e3d6a8eb3a12ec9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7e3d6a8eb3a12ed0 expref: -99 pid: 6518 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1782107183.558545:0:7431:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967319 is committed 00000020:00080000:0.0:1782107183.558558:0:7431:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa08b434c13c0 refcount 2 committed 0 result 0 batchid 5 00000020:00080000:0.0:1782107183.558562:0:7431:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffffa08b42fdc900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa08b434c1540 00000020:00080000:0.0:1782107183.558566:0:7431:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].5 00000020:00080000:0.0:1782107183.558568:0:7431:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffffa08b42fdc400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa08c7978f600 00000020:00080000:0.0:1782107183.558570:0:7431:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].5 00000020:00080000:0.0:1782107183.558593:0:7431:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa08c7a51a180 refcount 2 committed 0 result 0 batchid 4 00000020:00080000:0.0:1782107183.558596:0:7431:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffffa08c7bddf200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa08c7a51ac00 00000020:00080000:0.0:1782107183.558599:0:7431:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].4 00000020:00080000:0.0:1782107183.558601:0:7431:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffffa08c7bddf400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa08c51777200 00000020:00080000:0.0:1782107183.558604:0:7431:0:(update_trans.c:74:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].4 00000020:00080000:2.0:1782107183.558607:0:7453:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 5 committed batchid 1 00000020:00080000:0.0:1782107183.558618:0:7431:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0001: 3 batchid updated 00010000:00010000:3.0:1782107183.558693:0:3645:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558702:0:3645:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558709:0:3645:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558714:0:3645:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b42907c00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:3.0:1782107183.558718:0:3645:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558725:0:3645:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558753:0:3645:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffffa08c7a609400) 00010000:00010000:3.0:1782107183.558756:0:3645:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107183.558761:0:3645:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558767:0:3645:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b42907c00/0x7e3d6a8eb3a12ea6 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6517 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558806:0:3645:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558811:0:3645:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1782107183.558821:0:7453:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa08b42fd9e40 with version 10 00010000:00010000:3.0:1782107183.558826:0:3645:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558831:0:3645:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c412eaa00 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:3.0:1782107183.558834:0:3645:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1782107183.558839:0:7453:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 2 cancel update log [0x200000401:0x3:0x0].2: rc = 0 00010000:00010000:3.0:1782107183.558840:0:3645:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558849:0:3645:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5:0x0].0x0 (ffffa08c469af000) 00010000:00010000:3.0:1782107183.558852:0:3645:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107183.558854:0:3645:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558860:0:3645:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412eaa00/0x7e3d6a8eb3a12f40 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558869:0:3645:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558875:0:3645:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1782107183.558875:0:7453:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 2 cancel update log [0x240000400:0x3:0x0].2: rc = 0 00010000:00010000:3.0:1782107183.558881:0:3645:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1782107183.558895:0:7453:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa08b42fd96c0 with version 11 00010000:00010000:3.0:1782107183.558898:0:3645:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08c412ebc00 already unused, calling callback (ffffffffc1a094f0) 00000020:00080000:2.0:1782107183.558898:0:7453:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3 cancel update log [0x200000401:0x3:0x0].3: rc = 0 00010000:00010000:3.0:1782107183.558902:0:3645:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1782107183.558915:0:7453:0:(update_trans.c:1295:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3 cancel update log [0x240000400:0x3:0x0].3: rc = 0 00010000:00010000:3.0:1782107183.558917:0:3645:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1782107183.558925:0:7453:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3 00010000:00010000:3.0:1782107183.558926:0:3645:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0x0 (ffffa08c469af300) 00010000:00010000:3.0:1782107183.558929:0:3645:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107183.558932:0:3645:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558936:0:3645:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08c412ebc00/0x7e3d6a8eb3a12f47 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6518 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782107183.558958:0:3660:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b42fe6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269162752:0@lo:41:osp-pre-0-1.0 00010000:00010000:3.0:1782107183.558965:0:3645:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558970:0:3645:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558975:0:3645:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558979:0:3645:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa08b43418400 already unused, calling callback (ffffffffc1a094f0) 00010000:00010000:3.0:1782107183.558982:0:3645:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558987:0:3645:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.558993:0:3645:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x6:0x0].0xdd5a5043 (ffffa08c43d2b600) 00010000:00010000:3.0:1782107183.558995:0:3645:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1782107183.558998:0:3645:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782107183.559010:0:3645:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa08b43418400/0x7e3d6a8eb3a12fd3 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x6:0x0].0xdd5a5043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18105 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1782107183.559036:0:3661:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3661:1868674269162880:0@lo:103:jbd2/dm-1-8.0 00000004:00080000:0.0:1782107183.559087:0:7445:0:(osp_trans.c:1330:osp_get_next_request()) ou ffffa08c453a7840 version 10 rpc_version 10 00000020:00100000:0.0:1782107183.559119:0:7445:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1782107183.559128:0:7445:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa08c7bbf8700 x1868674269163136/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782107183.559310:0:7445:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7bbf8700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269163136:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1782107183.559691:0:3661:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107183.560011:0:6502:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269162880 00000100:00100000:1.0:1782107183.560018:0:6502:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269162880 00000100:00100000:1.0:1782107183.560085:0:6502:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 15 00000100:00100000:1.0:1782107183.560095:0:6502:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7ba52680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3661:x1868674269162880:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1782107183.560107:0:6502:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782107183.560150:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa08b43419a00/0x7e3d6a8eb3a12ed0 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7e3d6a8eb3a12ec9 expref: 7 pid: 7835 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782107183.560164:0:6502:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffffa08c43d2b300) 00010000:00010000:1.0:1782107183.560182:0:6502:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782107183.560221:0:6502:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7ba52680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3661:x1868674269162880:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 127us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107183.560228:0:6502:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 15 00000100:00100000:3.0:1782107183.560310:0:3661:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3661:1868674269162880:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1782107183.565897:0:7445:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107183.565947:0:7445:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08c81af0c00 going to sleep for 11 seconds 00000100:00100000:1.0:1782107183.566080:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269163136 00000100:00100000:1.0:1782107183.566090:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269163136 00000100:00100000:1.0:1782107183.566143:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:1.0:1782107183.566152:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c7bbf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7445:x1868674269163136:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1782107183.566444:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c7bbf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7445:x1868674269163136:12345-0@lo:1000:osp_up0-1.0 Request processed in 292us (552us total) trans 4294967364 rc 0/0 00000100:00100000:1.0:1782107183.566451:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:0.0:1782107183.568583:0:7445:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7bbf8700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269163136:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1782107183.568601:0:7445:0:(osp_trans.c:1330:osp_get_next_request()) ou ffffa08c453a7840 version 11 rpc_version 11 00000020:00100000:0.0:1782107183.568627:0:7445:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1782107183.568634:0:7445:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa08c7bbfad80 x1868674269163392/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782107183.568675:0:7445:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269163392:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1782107183.568728:0:7445:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107183.568774:0:7445:0:(client.c:2692:ptlrpc_set_wait()) set ffffa08c81af0c00 going to sleep for 11 seconds 00000100:00100000:3.0:1782107183.568838:0:6522:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269163392 00000100:00100000:3.0:1782107183.568844:0:6522:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269163392 00000100:00100000:3.0:1782107183.568903:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:3.0:1782107183.568911:0:6522:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b41567b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7445:x1868674269163392:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1782107183.569069:0:6522:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b41567b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7445:x1868674269163392:12345-0@lo:1000:osp_up0-1.0 Request processed in 159us (341us total) trans 4294967365 rc 0/0 00000100:00100000:3.0:1782107183.569077:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1782107183.569203:0:7445:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7bbfad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7445:1868674269163392:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1782107183.569859:0:3660:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782107183.571223:0:7835:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269162752 00000100:00100000:3.0:1782107183.571235:0:7835:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269162752 00000100:00100000:3.0:1782107183.571289:0:7835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:3.0:1782107183.571298:0:7835:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42fe5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3660:x1868674269162752:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1782107183.571333:0:7835:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446215680 avail 1299976192 00000100:00100000:3.0:1782107183.571464:0:7835:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42fe5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3660:x1868674269162752:12345-0@lo:41:osp-pre-0-1.0 Request processed in 167us (1605us total) trans 0 rc 0/0 00000100:00100000:3.0:1782107183.571470:0:7835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1782107183.572058:0:3660:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b42fe6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3660:1868674269162752:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1782107184.068248:0:3659:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1782107184.068269:0:3659:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08c7ba51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3659:1868674269163520:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782107184.068373:0:3659:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782107184.068501:0:17563:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269163520 00000100:00100000:1.0:1782107184.068510:0:17563:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269163520 00000100:00100000:1.0:1782107184.068582:0:17563:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:1.0:1782107184.068598:0:17563:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08c447db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3659:x1868674269163520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1782107184.068629:0:17563:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1782107184.068690:0:17563:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08c447db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3659:x1868674269163520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 94us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1782107184.068697:0:17563:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:1.0:1782107184.068745:0:3659:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08c7ba51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3659:1868674269163520:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1782107184.068797:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1782107184.071067:0:3659:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffffa08b41566680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3659:1868674269163648:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1782107184.071135:0:3659:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782107184.071221:0:8402:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1868674269163648 00000100:00100000:0.0:1782107184.071228:0:8402:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1868674269163648 00000100:00100000:0.0:1782107184.071284:0:8402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:0.0:1782107184.071292:0:8402:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffffa08b42fe6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3659:x1868674269163648:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1782107184.071329:0:8402:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1782107184.083498:0:8402:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffffa08b42fe6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3659:x1868674269163648:12345-0@lo:13:osp-pre-0-1.0 Request processed in 12195us (12351us total) trans 0 rc 0/0 00000100:00100000:0.0:1782107184.083521:0:8402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00080000:0.0:1782107184.083588:0:6512:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1782107184.087220:0:3659:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffffa08b41566680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3659:1868674269163648:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1782107184.134373:0:6504:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 1708 lines, 1708 kept, 0 dropped, 0 bad.